WO2025221093A1 - Transform or inverse transform method in geometric partitioning mode - Google Patents

Abstract

The present embodiment provides a transform or inverse transform method in a geometric partitioning mode. In the present embodiment, an image decoding device acquires dequantized transform coefficients of a current block. When the current block is predicted according to the geometric partitioning mode and indicates the use of a sub-block transform unit, the image decoding device determines the sub-block transform unit of the current block. The sub-block transform unit is determined on the basis of a prediction region including a weighted sum region of sub-regions in relation to the sub-regions of the current block partitioned according to the geometric partitioning mode. The image decoding device determines an inverse transform kernel of the sub-block transform unit and generates residual samples of the sub-block transform unit by applying the inverse transform kernel to the dequantized transform coefficients. The image decoding device uses the residual samples of the sub-block transform unit to fill the prediction region used for determining the sub-block transform unit<i />in a predetermined scanning order, and pads zero samples around the prediction region to generate a residual block of the current block.

Description

Transformation or inverse transformation method in geometric division mode

The present disclosure relates to a video coding method and apparatus for performing transformation or inverse transformation in a geometric partitioning mode.

The content described below merely provides background information related to the present invention and does not constitute prior art.

Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

Therefore, when storing or transmitting video data, the encoder compresses the video data and stores or transmits it, and the decoder receives the compressed video data, decompresses it, and plays it back. These video compression technologies include H.264/AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.

However, as image size, resolution, and frame rate continue to increase, and the amount of data to be encoded also increases, new compression technologies with better encoding efficiency and improved image quality are required. In particular, when the current block is predicted in geometric segmentation mode, a method for efficiently performing transformation or inverse transformation is required.

The present disclosure aims to provide a video coding method and device that efficiently performs transformation and inverse transformation on residual samples around a segmentation boundary when predicting a current block according to a geometric segmentation mode.

According to an embodiment of the present disclosure, a method for restoring a current block, performed by an image decoding device, comprises: obtaining inverse quantized transform coefficients of the current block; and, if the current block is predicted according to a geometric partitioning mode, decoding a flag from a bitstream indicating whether a sub-block transform unit is used, and if the flag indicates use of the sub-block transform unit, determining a sub-block transform unit of the current block, wherein the sub-block transform unit is determined based on a prediction region including a weighted sum region of sub-regions of the current block partitioned according to the geometric partitioning mode; determining an inverse transform kernel of the sub-block transform unit; applying the inverse transform kernel to the inverse quantized transform coefficients to generate residual samples of the sub-block transform unit; and, using the residual samples of the sub-block transform unit, performing a scanning operation according to a predetermined scanning order. A method is provided, further comprising a step of filling a prediction region used in determining the above sub-block transformation unit.

According to another embodiment of the present disclosure, a method for encoding a current block, performed by an image encoding device, comprises: obtaining residual samples of the current block; and, if the current block is predicted according to a geometric partitioning mode, obtaining a flag from a higher level indicating whether a sub-block transformation unit is not used, and, if the flag indicates use of the sub-block transformation unit, determining a prediction region used to configure a sub-block transformation unit of the current block, wherein the prediction region includes a weighted sum region of sub-regions of the current block divided according to a geometric partitioning mode; filling the sub-block transformation unit according to a predetermined scanning order using residual samples of the prediction region among residual samples of the current block; determining a transformation kernel of the sub-block transformation unit; and applying the transformation kernel to the sub-block transformation unit to generate transformation coefficients of the sub-block transformation unit.

According to another embodiment of the present disclosure, a method for providing video data to a video decoding device comprises: encoding the video data into a bitstream; and transmitting the bitstream to the video decoding device, wherein the encoding the video data comprises: obtaining residual samples of a current block; and, if the current block is predicted according to a geometric partitioning mode, obtaining a flag indicating whether a sub-block transformation unit is not used from a higher level; and, if the flag indicates use of the sub-block transformation unit, determining a prediction region used to configure a sub-block transformation unit of the current block, wherein the prediction region includes a weighted sum region of sub-regions of the current block divided according to a geometric partitioning mode; filling the sub-block transformation unit according to a predetermined scanning order using residual samples of the prediction region among residual samples of the current block; determining a transformation kernel of the sub-block transformation unit; And the method further includes a step of applying the transform kernel to the sub-block transform unit to generate transform coefficients of the sub-block transform unit.

As described above, according to the present embodiment, when predicting a current block according to a geometric segmentation mode, a video coding method and device are provided that efficiently perform transformation and inverse transformation on residual samples around a segmentation boundary, thereby making it possible to improve video encoding efficiency and enhance video quality.

FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.

Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

FIGS. 3A and 3B are diagrams illustrating multiple intra prediction modes, including wide-angle intra prediction modes.

Figure 4 is an example diagram of the surrounding blocks of the current block.

FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

FIG. 6 is a block diagram illustrating in detail a portion of an image decoding device according to one embodiment of the present disclosure.

FIG. 7 is an exemplary diagram showing sub-regions divided according to a geometric division mode according to one embodiment of the present disclosure.

FIG. 8 is an exemplary diagram showing a sub-block transformation unit according to one embodiment of the present disclosure.

FIG. 9 is an exemplary diagram showing a process for determining a sub-block transformation unit according to another embodiment of the present disclosure.

FIG. 10 is an exemplary diagram showing a reverse transformation process and a restoration process according to one embodiment of the present disclosure.

FIG. 11a and FIG. 11b are exemplary diagrams showing sub-block transformation units according to another embodiment of the present disclosure.

FIG. 12 is an exemplary diagram showing a reverse transformation process and a restoration process according to another embodiment of the present disclosure.

FIG. 13a and FIG. 13b are exemplary diagrams showing sub-block transformation units according to another embodiment of the present disclosure.

FIG. 14 is an exemplary diagram showing a reverse transformation process and a restoration process according to another embodiment of the present disclosure.

FIG. 15 is an exemplary diagram showing a process for determining an inverse transform kernel according to one embodiment of the present disclosure.

FIG. 16 is an exemplary diagram showing a process for determining an inverse transform kernel according to another embodiment of the present disclosure.

FIG. 17 is a flowchart illustrating a method by which an image encoding device transforms a transform block according to one embodiment of the present disclosure.

FIG. 18 is a flowchart illustrating a method for an image decoding device to inversely transform a transform block according to one embodiment of the present disclosure.

Hereinafter, embodiments of the present invention will be described in detail with reference to exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, in describing the present embodiments, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present embodiments.

FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to the illustration in FIG. 1.

The video encoding device may be configured to include a picture segmentation unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).

Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and/or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). And, information commonly referenced by one or more SPS is encoded in a Video Parameter Set (VPS). In addition, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile or tile group header may be referred to as high level syntax.

The picture segmentation unit (110) determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as the syntax of SPS or PPS and transmitted to the image decoding device.

The picture segmentation unit (110) divides each picture constituting an image into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU, which is a basic unit of encoding.

The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).

Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.

As illustrated in FIG. 2, a CTU may first be split into a QT structure. The quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it may be further split into one or more of the BT structure or the TT structure. There may be multiple splitting directions in the BT structure and/or the TT structure. For example, there may be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As illustrated in FIG. 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and/or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is a basic unit of encoding. If the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding device starts encoding from the first flag in the above-described manner.

As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.

A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.

The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

In general, each current block within a picture can be predictively coded. Prediction of the current block can typically be performed using either intra-prediction (using data from the picture containing the current block) or inter-prediction (using data from a picture coded before the picture containing the current block). Inter-prediction encompasses both unidirectional and bidirectional prediction.

The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3a, the multiple intra prediction modes may include two non-directional modes including the Planar mode and the DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently depending on each prediction mode.

For efficient directional prediction for a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in Fig. 3b may be additionally used. These may be referred to as "wide-angle intra-prediction modes." In Fig. 3b, the arrows point to corresponding reference samples used for prediction, and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide-angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. At this time, among the wide-angle intra-prediction modes, some wide-angle intra-prediction modes available for the current block may be determined based on the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having an angle less than 45 degrees are available when the current block is a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available when the current block is a rectangular shape whose width is larger than its height.

The intra prediction unit (122) can determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate bit-rate distortion values using rate-distortion analysis for multiple tested intra prediction modes and select an intra prediction mode with the best bit-rate distortion characteristics among the tested modes.

The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector (MV) corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

The inter prediction unit (124) may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision or resolution of the motion vector can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. Information on the motion vector resolution may be information indicating the precision of a differential motion vector, which will be described later.

Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the entropy encoding unit (155). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.

Various methods can be used to minimize the number of bits required to encode motion information.

For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the image decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode'.

In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

As the surrounding blocks for deriving merge candidates, all or part of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the block at the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, the merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the video decoding device.

Merge Skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high encoding efficiency can be achieved for low-motion images, still images, and screen content images.

Hereinafter, merge mode and merge skip mode are collectively referred to as merge/skip mode.

Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.

In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block in the current picture as shown in FIG. 4 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block co-located with the current block in the reference picture or blocks adjacent to the block in the co-located block may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.

The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information to identify the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.

The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.

The transformation unit (140) transforms residual signals within a residual block having pixel values in a spatial domain into transform coefficients in a frequency domain. The transformation unit (140) may transform the residual signals within the residual block using the entire size of the residual block as a transformation unit, or may divide the residual block into a plurality of sub-blocks and use the sub-blocks as transformation units to perform the transformation. Alternatively, the residual signals may be transformed using only the transformation domain sub-block as a transformation unit by dividing the sub-blocks into two sub-blocks, that is, a transformation domain and a non-transform domain. Here, the transformation domain sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only a sub-block has been converted, directionality (vertical/horizontal) information (cu_sbt_horizontal_flag), and/or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. In addition, the size of the conversion area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

Meanwhile, the transformation unit (140) can individually perform transformations on the residual block in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation function pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation function pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.

The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.

The rearrangement unit (150) can perform rearrangement of coefficient values for quantized residual values.

The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence by using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region by using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, the scanning method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (information about an encoding mode of motion information (merge mode or AMVP mode), a merge index in the case of a merge mode, and a reference picture index and a differential motion vector in the case of an AMVP mode) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about a quantization parameter and information about a quantization matrix.

The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

An adder (170) adds the restored residual block and the predicted block generated by the prediction unit (120) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting the next block.

The loop filter unit (180) performs filtering on restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation/quantization. The loop filter unit (180) may include all or part of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186) as an in-loop filter.

The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding/decoding, and the SAO filter (184) and the ALF (186) perform additional filtering on the deblocking-filtered image. The SAO filter (184) and the ALF (186) are filters used to compensate for the differences between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in units of CTUs. In contrast, the ALF (186) performs block-based filtering, and compensates for distortion by applying different filters by distinguishing the edges and degrees of variation of the corresponding block. Information on filter coefficients to be used in the ALF can be encoded and signaled to an image decoding device.

The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

The video encoding device can store the bitstream of encoded video data on a non-transitory storage medium or transmit it to the video decoding device using a communication network.

FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.

The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information, information on residual signals, etc. required to restore the current block.

The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.

For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (mtt_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and/or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Accordingly, each node below the leaf nodes of QT are split recursively into the BT or TT structure.

As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.

As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.

Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded by using the division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts syntax elements for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about residual signals.

The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.

The inverse quantization unit (520) inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using the quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.

The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a residual block for the current block.

In addition, when the inverse transform unit (530) inversely transforms only a portion of a transform block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only a sub-block of the transform block has been transformed, directionality (vertical/horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and/or position information (cu_sbt_pos_flag) of the sub-block, and inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore residual signals, and fills “0” values with residual signals for areas that have not been inversely transformed, thereby generating a final residual block for the current block.

In addition, when MTS is applied, the inverse transform unit (530) determines a transform function or a transform matrix to be applied in the horizontal and vertical directions using MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.

The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.

The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

The inter prediction unit (544) uses the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.

An adder (550) adds the residual block output from the inverse transform unit (530) and the predicted block output from the inter prediction unit (544) or the intra prediction unit (542) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting a block to be decoded later.

The loop filter unit (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as in-loop filters. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) and the ALF (566) perform additional filtering on restored blocks after deblocking filtering to compensate for differences between restored pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about filter coefficients decoded from the non-stream.

The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within a picture to be encoded later.

The present embodiment relates to encoding and decoding of images (videos) as described above. More specifically, the present invention provides a video coding method and device that efficiently performs transformation and inverse transformation on residual samples around a segmentation boundary when predicting a current block according to a geometric segmentation mode.

The following embodiments may be performed by a prediction unit (120), a transformation unit (140), and an inverse transformation unit (165) within a video encoding apparatus. In addition, the following embodiments may be performed by a prediction unit (540) and an inverse transformation unit (530) within a video decoding apparatus.

The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization in encoding the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoding device. The video decoding device can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoding unit (510).

In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU). Alternatively, the term "target block" may also refer to a portion of a coding unit.

Also, a value of a flag being true indicates that the flag is set to 1. Also, a value of a flag being false indicates that the flag is set to 0.

The decoder side includes all or part of the inverse quantization unit (160), the inverse transform unit (165), the prediction unit (120), the adder (170), the loop filter unit (180), and the memory (190) in the video encoding device illustrated in FIG. 1. Alternatively, the decoder side includes all or part of the inverse quantization unit (520), the inverse transform unit (530), the prediction unit (540), the adder (550), the loop filter unit (560), and the memory (570) in the video decoding device illustrated in FIG. 5. With respect to a series of decoding processes, the decoder side of the video encoding device and the decoder side of the video decoding device perform the same operation. The video encoding device determines information related to the operation of the decoder side and signals the determined information to the video decoding device. The video decoding device can decode the signaled information and operate the decoder side based on the decoded information.

I-1. Geometric Partitioning Mode (GPM)

The GPM of VVC obtains a partitioning mode for partitioning the current block into two sub-regions from among 64 predefined geometric partitioning modes based on information extracted from the bitstream, and motion information for each region. The GPM performs motion compensation of the two sub-regions according to the motion information, and weights the predictors of the two sub-regions using a blending matrix (i.e., a weight matrix) according to the partitioning mode to generate a final prediction block of the current block. In the ECM (Enhanced Compression Model), which corresponds to the next-generation VVC, a technology for generating a prediction block according to intra prediction in one or more of the two sub-regions, a technology for generating a prediction block according to IBC (Intra Block Copy), a technology for generating a prediction block according to IntraTMP, and a technology for generating a prediction block according to affine prediction are added to the GPM mode.

At this time, the technology in which both sub-regions generate prediction blocks according to intra prediction is called SGPM (Spatial GPM). When the two sub-regions are referred to as sub-region 0 and sub-region 1, respectively, whether intra prediction is applied to sub-region 0 is extracted from the bitstream. If intra prediction is not applied to sub-region 0, whether intra prediction is applied to sub-region 1 is extracted. At this time, if intra prediction is applied to sub-region 0, a prediction block can be generated in sub-region 1 according to motion compensation.

In ECM, the GPM mode can perform MMVD (Merge with Motion Vector Difference) on each sub-region and perform motion information compensation based on Template Matching (TM). For a single CU, only one GPM-MMVD and one GPM-TM can be applied. GPM-MMVD can be applied independently to each sub-region.

In ECM, GPM mode can variably consider the weighted sum area by signaling/parsing information related to the weighted sum area as an index and using the index in the process of obtaining the weighted sum matrix.

In ECM, BI-GPM mode can generate a prediction block in each sub-region based on bidirectional prediction during inter prediction. That is, the Bi-GPM technology can generate a prediction block in each sub-region based on bidirectional prediction when generating a prediction block of the current block based on the GPM technology. The Bi-GPM technology is applicable to all GPM-predicted blocks except for blocks of 8×8, 8×16, and 16×8 sizes. With respect to the aforementioned small-sized blocks, the Bi-GPM technology generates a GPM merge list from the general merge list such that the list candidates have unidirectional motion information. On the other hand, with respect to the aforementioned small-sized blocks and blocks of different sizes, the Bi-GPM technology generates a general merge list and generates a prediction block in each sub-region based on merge index information extracted from the bitstream. In addition, with respect to the Bi-GPM technology, bidirectional GPM-MMVD and bidirectional GPM-TM are supported, and a BDOF (Bi-directional Optical Flow)-based motion vector enhancement technology can be applied to the sub-regions.

Implicit GPM technology does not signal or parse information about geometric segmentation boundaries. Implicit GPM technology derives weights for each location by applying regression analysis to the relationship between the predicted blocks of two sub-regions, the templates of the two predicted blocks, and the template of the current block. The derived weights are then used to generate the final predicted block. Affine prediction cannot be used when predicting each sub-region. Furthermore, the merge indices of the sub-regions are managed as a single combination, meaning that only one index indicating the combination is signaled. If both sub-regions are inter-predicted, TM (Template Matching)-based motion information compensation can be performed on the motion vectors of each sub-region. This compensation information can be extracted from the bitstream. Furthermore, intra prediction can be used in at least one of the sub-regions. Information regarding the use of intra prediction can be extracted from the bitstream. If this information is true, intra prediction can be included in the prediction mode candidate list combination of the sub-regions.

I-2. Conversion Technology - Primary Conversion Technology

As mentioned above, for efficient video compression, quantization or scaling can be additionally applied to residual signals remaining after prediction using various prediction techniques. At this time, a transform technique can be applied based on the importance of perceptual visual information inherent in the residual signals to cluster the residual signals into one group according to frequency components, and then scaling can be performed. However, for non-natural signals, such as screen contents, this frequency-based transform technique can be inefficient. In such cases, the transform technique can be omitted, and only scaling can be performed, or encoding/decoding can be performed without applying scaling.

When a transform is applied in HEVC, DCT-II is used as a transform kernel (hereinafter, used interchangeably with transform type) to transform residual signals. However, in order to apply a more appropriate transform technique depending on the diversity of residual signal characteristics, Multiple Transform Selection (MTS) can be used. MTS determines the optimal type among multiple transform types, and then transforms the block according to the determined transform type. For example, in VVC, as shown in Table 1, in addition to DCT-II, two other transform types, DCT-VIII and DST-VII, are added so that residual signals can be transformed in various ways.

Here, basis functions form a transformation matrix that defines each transformation type. Hereinafter, DCT-II, DCT-VIII, and DST-VII are used interchangeably with DCT2, DCT8, and DST7, respectively.

Meanwhile, the flag determining whether MTS is used can be controlled at the block level. Furthermore, MTS use can be controlled using an activation flag at the higher SPS level.

When MTS is enabled in SPS, a CU level flag may be displayed to indicate whether MTS is applied. Here, MTS may be applied to the luma component. The CU level flag may be expressed when both the width and height of the Transform Block (TB) are less than or equal to 32 pixels, and the Coded Block Flag (CBF), which indicates whether any of the transform coefficient levels have non-zero values, is true.

When the CU level flag is 0, DCT2 is used as a bidirectional kernel in both the horizontal and vertical directions. On the other hand, when the CU level flag is not 0, MTS is applied. MTS can be used in two ways: explicit MTS and implicit MTS.

In explicit MTS, the kernel used for the TB is explicitly transmitted. Typically, the index of the transformation kernel can be transmitted. For example, the kernel index, mts_idx, can be defined as shown in Table 2.

Here, trTypeHor and trTypeVer represent the horizontal and vertical transformation types, respectively. Additionally, 0 represents DCT2, 1 represents DST7, and 2 represents DCT8.

Meanwhile, in implicit MTS, for example, the transformation type can be determined implicitly even if MTS is not explicitly signaled in the case of an intra block. In VVC, the horizontal and vertical transformation types can be determined implicitly, as in Equation 1.

Here, nTbW and nTbH represent the horizontal and vertical lengths of the transformation block, respectively.

Meanwhile, the next-generation technology, ECM (Enhanced Compression Model) software, increases the number and types of MTS kernels, adding DST7, DCT8, DCT5, DST4, DST1, and identity transform.

I-3. Low-frequency Non-separable Transform (LFNST)

The LFNST technique applies a secondary transform to the low-frequency region among the transform coefficients generated according to the primary transform of the transform unit (TU) during intra prediction. In terms of encoding, the LFNST technique applies the secondary transform to L low-frequency primary transform coefficients among the W×H primary transform coefficients, thereby generating K (where K ≤ L) secondary transform coefficients. Here, the size of the transform kernel of the LFNST is L×K. That is, the LFNST technique expresses the L low-frequency primary transform coefficients among the W×H primary transform coefficients as a 1×L vector, and then applies the L×K transform kernel to generate a 1×K vector. Thereafter, the LFNST technique expresses the 1×K vector as a two-dimensional array in the low-frequency region for subsequent processes such as quantization.

Compared to the first-order transformation that applies separate transformation kernels in the horizontal and vertical directions, the LFNST technique performs a non-separable transform that transforms a one-dimensional vector.

Meanwhile, the type of transformation kernel can be determined based on the intra prediction mode of the current TU, the size of the TU, and the LFNST index (lfnst_idx). For example, in VVC, a set of transformation kernels can be determined based on the intra prediction mode (IntraPredMode) of the current TU, as shown in Table 3.

Here, the intra prediction mode (IntraPredMode) follows the example of Fig. 3b. Additionally, lfnstTrSetIdx is an index indicating a kernel set. In Table 3, IntraPredMode values of 81, 82, and 83 indicate CCLM (Cross-component Linear Model) prediction modes.

For each kernel set (lfnstTrSetIdx), two types of kernels are defined. Which kernel is selected among the two types of kernels can be indicated by the LFNST index. If the LFNST index is 0, it means that LFNST is not performed, and if the LFNST index is 1 or 2, a different LFNST kernel within the same kernel set is applied. Since there is one more kernel set depending on the size of the TU, there are a total of 4×2×2 = 16 transformation kernels. The kernel sizes of LFNST are defined as 16×16 and 16×48. In addition, the kernel size can be adjusted as shown in Table 4 depending on the size of the TU.

In ECM, a set of transformation kernels can be defined depending on the block shape and intra prediction mode. For example, by utilizing the symmetry property of square blocks, a set of 35 transformation kernels is defined for a square block depending on the intra prediction mode.

Meanwhile, when MIP (Matrix-weighted Intra Prediction) prediction is performed on the current TU in VVC, the image decoding device maps the intra prediction mode of the current TU to the Planar mode and then determines the transformation kernel according to the mapped prediction mode. The MIP technology includes MIP boundary downsampling, Matrix-Vector multiplication, and MIP prediction upsampling processes. 'MIP boundary downsampling' refers to the process of downsampling a specific number of pixels among the boundary pixels of the current block. 'Matrix-Vector multiplication' refers to the process of generating a reduced prediction block by multiplying the downsampled pixels (i.e., 'vector') by a preset matrix. 'MIP prediction upsampling' refers to the process of generating a final prediction block by upsampling the reduced prediction block.

In addition, when MIP prediction is performed in the next-generation technology, ECM (Enhanced Compression Model), the video decoding device obtains a directional prediction mode by applying DIMD (Decoder-side Intra Mode Derivation) to the MIP-predicted block, i.e., the output of the 'Matrix-Vector Product'. The video decoding device maps the intra prediction mode of the current TU to the obtained directional prediction mode, and then determines the transformation kernel according to the mapped prediction mode. The DIMD technology calculates the gradient of each sample for adjacent samples of the current block, generates a HoG (Histogram of Gradient) from the calculated gradients, and derives a prediction mode for intra prediction of the current block according to the generated HoG.

Meanwhile, when the DCT2/DCT2 transform kernel is applied as a first transform to the intra-predicted TU in VVC, the LFNST technique can be applied as a second transform.

In ECM, when the block size of the TUs encoded according to intra prediction is small (e.g., 4×4, 4×8, 8×4, 4×16, 16×4, 8×8, 8×16, 16×8), the non-separable first-order transform technique can be applied. In addition, when the block size is large and the DCT2/DCT2 transform kernel is applied to the block as the first-order transform, the LFNST technique can be applied.

I-4 Non-separable Primary Transform (NSPT)

The non-separable first-order transform technique performs a non-separable first-order transform using a pre-learned transform kernel instead of performing vertical and horizontal transforms on the residual signals of blocks predicted in the intra prediction mode.

NSPT technology uses non-separable primary transformation kernels of different sizes depending on the size of the residual block. For example, if the size of the transformation block is 4×4, a 16×16 kernel is applied to generate 4×4 transformation coefficients. As another example, if the size of the transformation block is 4×8, a 32×20 kernel is applied to generate 20 transformation coefficients. The generated transformation coefficients are arranged in the transformation block according to the promised scanning order. If the number of transformation coefficients generated after the transformation is less than the number of pixels in the transformation block, the remaining area where the transformation coefficients are not arranged is filled with 0.

For non-separable first-order transformations, transformation kernels can be defined based on the block shape, intra prediction mode, and NSPT index (nspt_idx). For example, by leveraging the symmetry of square blocks, there are 35 sets of transformation kernels for a square block, each with its own intra prediction mode. The NSPT index (i.e., the LFNST index) can be used to indicate one of the candidate kernels within the transformation kernel set.

A non-separable first-order transform can be applied to the luma component. Furthermore, if a non-separable first-order transform is applied, the second-order transform, i.e., LFNST, may not be performed.

The following embodiments are described with a focus on a video decoding device, but can be implemented in the same or similar manner in a video encoding device. Alternatively, the following embodiments are described with a focus on the decoder side of a video decoding device, but can also be implemented in the same or similar manner in the decoder side of a video encoding device.

II. Embodiments according to the present disclosure

FIG. 6 is a block diagram illustrating in detail a portion of an image decoding device according to one embodiment of the present disclosure.

The video decoding device according to the present embodiment determines prediction and transformation units, and performs prediction and inverse transformation on the current block corresponding to the determined unit using the determined prediction technique and prediction mode, thereby finally generating a restoration block of the current block. The example illustrated in FIG. 6 may be performed by the inverse transformation unit (530), the prediction unit (540), and the adder (550) of the video decoding device. Meanwhile, the same operations as the example illustrated in FIG. 6 may be performed by the inverse transformation unit (165), the picture division unit (110), the prediction unit (120), and the adder (170) of the video encoding device. At this time, the video decoding device uses encoding information parsed from the bitstream, but the video encoding device may use encoding information set from a higher level in terms of minimizing rate distortion. Hereinafter, for convenience, the present embodiment will be described with reference to the video decoding device.

As shown in the example of FIG. 5, the prediction unit (540) includes an intra prediction unit (542) and an inter prediction unit (544) depending on the prediction technology, but as shown in FIG. 6, the prediction unit (540) may include all or part of the prediction unit determination unit (602), the prediction technology determination unit (604), the prediction mode determination unit (606), and the prediction execution unit (608).

The prediction unit determination unit (602) determines a prediction unit (PU). The prediction technique determination unit (604) determines a prediction technique (e.g., intra prediction, inter prediction, IBC (Intra Block Copy) mode, palette mode, a technique that mixes intra and inter prediction, etc.) for the prediction unit. The prediction mode determination unit (606) determines a detailed prediction mode for the prediction technique. The prediction execution unit (608) generates a prediction block of the current block according to the determined prediction mode.

The inverse transform unit (530) includes all or part of the inverse transform unit determination unit (610), the inverse transform kernel determination unit (612), and the inverse transform execution unit (614). The inverse transform unit determination unit (610) determines a transformation unit for the inverse quantization signals (i.e., inverse quantization transform coefficients) of the current block. The inverse transform kernel determination unit (612) determines an inverse transform kernel, and the inverse transform execution unit (614) inversely transforms the transformation unit expressed by the inverse quantization transform coefficients, thereby generating residual samples.

Hereinafter, the transform unit (TU) on the encoding side can be used interchangeably with the transform block. The inverse transform unit and inverse transform block on the decoding side correspond to the TU and transform block, respectively. Therefore, the inverse transform unit and inverse transform block can be used interchangeably with the TU and transform block.

A transform unit is a unit that determines whether to perform an inverse transform with respect to transform coefficients and represents a unit in which information about the inverse transform is transmitted. An image decoding device determines a kernel to be applied to a transform unit and performs an inverse transform of transform coefficients based on the determined kernel. At least one kernel may be determined for one transform unit. According to an embodiment, an inverse transform of N orders may be applied, and the sizes of the transform unit and the inverse transform kernel in each order may not be the same. For example, when an inverse transform of N orders is applied to a current transform unit block (hereinafter, transform block or current transform block), the sizes of the kernels used for the inverse transform of each order may not be the same. That is, the image decoding device may apply an inverse transform to some of the input transform coefficients.

An adder (550) adds a prediction block and residual samples to generate a restoration block. The restoration block is stored in memory and can be used to predict other blocks.

If the color format of the input video is a YUV format (such as YUV420, YUV411, YUV422, YUV444), the video decoding device can perform prediction and restoration of the chroma component after performing prediction and restoration of the luma component. That is, the luma component and the chroma component can be sequentially restored by the components illustrated in Fig. 6. Meanwhile, if the color format of the input video is RGB, the video encoding device can perform color format conversion from RGB to YUV and then encode the converted video. Here, in the case of the YUV format, the color format represents the correspondence between the pixels of the luma component and the pixels of the chroma component.

The prediction unit determination unit (602) determines the size and shape of the prediction target block, and may utilize all direct or indirect information transmitted from the video encoding device. The prediction unit determination unit (602) may utilize direct information related to the size and shape of the current block, and may utilize information that may influence the determination of the size and shape of the current block, such as the number of divisions, depth, shape of division, direction of division, size information related to the minimum division block, and division information/prediction mode of the decoded neighboring blocks.

The prediction unit determined by the prediction unit determination unit (602) may be a current block, one of the sub-blocks into which the current block is divided, a set of pixels, or a single pixel. The prediction unit may include size information and shape information for performing prediction of chroma components and luma components.

The prediction unit can be determined dependently or independently for the chroma component and the luma component. Dependent determination means that the prediction unit of the luma component or chroma components is not determined for each component, but rather, when the prediction unit of one component is determined, the units of the other components or components are determined with a corresponding size and shape. In this case, one component may correspond to one or more of the luma component and the chroma component. That is, the luma component may be determined based on information of the chroma components. For example, the prediction unit of the chroma component may have a size corresponding to the prediction unit of the luma component depending on the color format. In the case of dependent determination, information about the prediction unit of the other component corresponding to one component, i.e., the dependently determined component, may be omitted. Independent determination means that the prediction units of the luma component and the chroma component are determined separately. In the case of independent determination, information about the prediction unit of each component may be signaled separately.

The prediction technique determination unit (604) determines a prediction technique for each prediction unit. As described above, the prediction technique may be one of inter prediction, intra prediction, IBC mode, palette mode, or a technique combining intra and inter prediction. In this case, the prediction technique for the chroma component may be determined in the same manner as the prediction technique for the corresponding luma component, without separate signaling or parsing of information. In some embodiments, a technique combining intra and inter prediction may be included in inter prediction.

For example, if the prediction technique of the current block is not intra prediction, the video decoding device parses 1-bit flag information. For example, if the parsed flag indicates Skip mode, the video decoding device determines the prediction mode of the current block as the merge mode of inter prediction or the IBC merge mode. In the case of Skip mode, the video decoding device can use the prediction signals as restored signals without performing the inverse transformation process (i.e., without parsing the residual signals).

On the other hand, if the parsed flag does not indicate a Skip mode for the current block, the prediction technique determination unit (604) can parse a series of 1-bit flags to determine the prediction technique of the current block as one of techniques such as inter prediction, intra prediction, IBC mode, palette mode, etc.

For example, if Skip is not applied to the current block and the prediction technique is determined to be Inter-Prediction or IBC mode, the video decoding device parses a 1-bit flag. Depending on the parsed flag, the prediction mode of the current block can be determined as either General Merge mode or Advanced Motion Vector Prediction (AMVP) mode.

The prediction mode determination unit (606) determines a detailed prediction mode of the current prediction unit block (hereinafter, used interchangeably with the current block) in relation to the prediction technology.

For example, if the prediction technology of the current block is inter prediction, the prediction mode determining unit (606) may determine the general merge mode or AMVP mode as the prediction mode of the current block. In the general merge mode or AMVP mode, the image decoding device generates prediction blocks according to one or more motion compensations based on parsed motion information, and weights and combines the generated multiple prediction blocks to generate final prediction signals of the current block. At this time, one or more of the prediction blocks may include a signal of a decoded area within a frame including the current block.

As another example, if the prediction technique for the current block is inter-prediction and intra-predicted samples are used for the final prediction block of the current block, the surrounding reconstructed region of the current block can be defined as a template, and an intra-prediction mode can be derived using this template. The template can also include regions not adjacent to the current block. The final prediction block of the current block can then be generated using the derived intra-prediction mode.

As another example, if the prediction technology of the current block is inter prediction, the prediction mode determining unit (606) may determine a geometric partitioning-based prediction mode (hereinafter, geometric partitioning mode (GPM)) as the prediction mode of the current block. In GPM, the image decoding device divides the current block into two or more sub-regions according to geometric partitioning, generates prediction blocks according to one or more motion compensations based on the motion information and prediction mode information of the parsed current block, and weights and combines the generated plurality of prediction blocks to generate final prediction signals of the current block. According to an embodiment, if inter prediction is used and the geometric partitioning mode is determined, at least one block among the sub-blocks in the current block may be predicted according to inter prediction.

Meanwhile, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a mode that generates a prediction block of the current block based on at least one of a directional prediction mode, a planar mode (Horizontal Planar, Vertical Planar, or Regular Planar), a DC mode, an EIP (Extrapolation intra prediction) prediction mode, a matrix-based prediction mode (for example, MIP), or a prediction mode based on correlation between components (for example, CCLM (Cross Component Linear Model), CCCM (Convolutional Cross Component Model), GLM (Gradient Linear Model), etc.).

As an example, if the prediction technology of the current block is intra prediction, the prediction mode of the current block may be a matrix-based intra prediction mode (for example, MIP (Matrix-based intra prediction)). The MIP mode may signal/parse and/or derive an index of a matrix based on a predefined matrix according to an agreement between an image encoding device and an image decoding device, or may signal/parse and/or derive a matrix and generate a prediction block of the current block based on the matrix.

As another example, if the prediction technique for the current block is intra prediction, the prediction mode for the current block may be intra-template matching prediction (IntraTMP). IntraTMP mode defines a restored area surrounding the current block as a template, and performs template matching on the restored area surrounding the current block to generate a predicted block. The template may also include areas not adjacent to the current block.

As another example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be an intra geometric segmentation-based prediction mode. The intra geometric segmentation-based prediction mode divides the current block into one or more sub-regions according to geometric segmentation, generates a prediction block of each region using intra prediction modes including different directional prediction modes, Planar mode, DC mode, etc., and weights each prediction block to generate a prediction block of the current block. According to an embodiment, if the prediction technique is intra prediction and the geometric segmentation-based prediction mode is determined, all sub-blocks within the current block may be predicted according to intra prediction.

As another example, if the current block is a chroma block and the prediction technique of the current block is intra prediction, the current block can be predicted according to DM (Direct Mode). DM mode can perform prediction of the current chroma block based on a prediction method applied to a luma block corresponding to the current chroma block, or a prediction method predefined between a video encoding device and a video decoding device.

As another example, the relationship between the surrounding restored chroma samples of the current chroma component and the surrounding restored luma samples of the luma region at the location corresponding to the current chroma block may be calculated using a linear and/or nonlinear model, and a prediction block of the current chroma block may be generated based on one or more of the calculated models.

As another example, there may be various methods for deriving the intra prediction mode of the current block. Among the multiple methods, one or more may be used based on the signaling/parsing of flags and/or indices.

Meanwhile, if the prediction technology of the current block is IBC prediction, a prediction block can be obtained from a previously restored area within a frame including the current block using one or more block vectors, and a final prediction block can be generated based on the prediction block. The video encoding device can signal information of the block vector, and the video decoding device can parse information of the block vector. For example, the video decoding device can construct a block vector candidate list according to a position and search order defined according to an agreement between the video encoding device and the video decoding device, and parse information such as an index. The video decoding device can obtain final block vector information by correcting the initial block vector information obtained based on the parsed information using a method such as template matching.

For example, if the prediction technique of the current block is IBC prediction, the prediction mode of the current block may be an IBC geometric partitioning-based prediction mode (hereinafter, IBC geometric partitioning mode). The IBC geometric partitioning mode may divide the current block into one or more sub-regions according to the geometric partitioning, generate a prediction block for each region using a different block vector, and weight and combine the prediction blocks to generate a final prediction block of the current block. According to an embodiment, if the current block is IBC prediction and the geometric partitioning mode is determined, at least one of all sub-blocks within the current block may be predicted according to the IBC prediction.

As another example, if the current block is a chroma block and the prediction technology of the current block is IBC prediction, and the corresponding luma block is restored during the process of constructing a block vector candidate list or obtaining a block vector, the block vector information of the luma block at the corresponding position can be used. For example, if the block division structures of the luma component and the chroma component are the same, the block vector information of the corresponding luma block can be scaled according to a color format to generate information, and the scaled information can be used as the block vector and/or block vector candidate of the current chroma block. On the other hand, if the block division structures of the luma component and the chroma component are different, one or more block vectors can be acquired according to a predefined position and order within the corresponding luma region. The acquired block vector can be scaled according to a color format to generate information, and the scaled information can be used as the block vector and/or block vector candidate of the current chroma block.

Meanwhile, if the prediction technology of the current block is a mixed technology of intra and inter prediction, the surrounding restoration area of the current block may be defined as a template, and an intra prediction mode may be derived using information of some or all pixels of the template, or an intra prediction block may be generated based on template matching. The prediction block of the current block may be derived according to the derived intra prediction mode, or the prediction block may be generated based on block vector information signaled from a video encoding device. The final prediction block may be generated by mixing the prediction block/intra prediction block, or inter prediction block generated based on template matching and/or block vector, according to a method such as weighted summation.

A template may include both adjacent and non-adjacent regions to the current block. The non-adjacent region may be an region within a certain distance of pixel lines from the current block. When a non-adjacent region is used as a template, information on whether it is used and/or the distance may be transmitted from the video encoding device to the video decoding device. For example, the distance information may be defined according to an agreement between the video encoding device and the video decoding device, and the transmission of the information may be omitted. If defined according to an agreement, the value may be fixed to a specific constant or may be variably determined based on the horizontal and vertical pixel lengths of the prediction unit, the block width, the aspect ratio, etc. Thereafter, the final prediction block of the current block may be generated using the derived intra prediction mode.

The prediction execution unit (608) generates a final prediction block of the current decryption block (hereinafter, used interchangeably with the current block) according to the determined prediction technology and prediction mode.

As an example, the prediction performing unit (608) performs prediction according to the prediction mode determined as described above to generate a prediction block of the current block, and the adder (550) adds the prediction block of the current block and residual samples (i.e., residual block) to generate a restoration block.

For example, if the prediction mode of the current block is a geometric segmentation mode, the image decoding device may parse and/or derive geometric segmentation information related to the current block, and classify the current block into multiple sub-regions based on the geometric segmentation information. Depending on the embodiment, the geometric segmentation may be performed in a non-linear form, and the number of sub-regions may be 2 or more. For example, it may be divided into sub-regions P0 and P1, as in the example of FIG. 7.

The video decoding device can generate a prediction block for each sub-region and weight the prediction blocks of the sub-regions to generate a final prediction block of the current block. For example, as shown in FIG. 7, when the number of sub-regions is 2, the prediction blocks of the sub-regions can be weighted and combined according to Equation 2.

Here, P ₀ (ij) and P ₁ (ij) represent prediction blocks of the P 0 and P 1 sub-regions, respectively, and n represents an integer greater than or equal to 1. As an example, w (i, j) may be a weighted sum matrix derived in an image decoding device based on geometric segmentation information of the current block, the size of the current block, aspect ratio, etc.

The weighted sum matrix can determine the region where the prediction samples of the P0 and P1 sub-regions coexist based on the geometric segmentation boundary, that is, the region where the prediction samples of the two sub-regions are weighted-summed according to a non-zero coefficient. At this time, the region to be weighted-summed can be determined as a fixed region according to geometric segmentation boundary information and the size of the current block. The video encoding device can signal region area information, and the video decoding device can parse the region area information and determine the weighted sum matrix by considering the region area information together. The region area information can be defined in the form of a table according to an agreement between the video encoding device and the video decoding device, and can be determined according to the index of the table. At this time, information related to the table (e.g., the size of the table, the region area information included in the table) can vary depending on the size and aspect ratio of the current block, etc.

For example, if the prediction mode of the current block is a geometric segmentation mode and an intra prediction mode other than IntraTMP is not used to generate the final prediction block, the image decoding device may derive coefficients of a weighted sum matrix using the template of the current block and templates of a plurality of reference blocks without parsing and/or deriving geometric segmentation information for the current block, and may generate the final prediction block using the derived coefficients. In this case, affine prediction may not be used. In this case, affine prediction may be a mode determined by signaling/parsing a 1-bit flag in addition to the geometric segmentation mode flag.

As another example, assume that the prediction mode of the current block is geometric partitioning mode and the number of sub-regions is 2 or more. When generating the final prediction block of the current block using the prediction block of each sub-region, the prediction techniques for generating each prediction block can be at most N types (e.g., when N is 2, the prediction techniques can be inter prediction and intra prediction). As an example, when the number of sub-regions is 4 and N is 2, only intra prediction and inter prediction can be used to perform prediction of the current block. As another example, when the number of sub-blocks is X and N is 2, and both intra prediction and inter prediction are available, all sub-blocks can be predicted only by intra prediction or only by inter prediction to perform prediction of the current block.

As another example, assume that the prediction mode of the current block is geometric segmentation-based prediction mode, and at least one sub-region is predicted using inter-prediction, inter-block prediction, or intra-temporal prediction. In this case, after generating prediction blocks for the sub-regions, illumination compensation may be performed.

For example, an image decoding device can derive a model for illumination compensation using the relationship between the template of a prediction block of each sub-region and the template of the corresponding current block, and compensate prediction samples of the sub-region based on the derived model, thereby generating a compensated prediction block. The illumination compensation can be applied to a sub-region in which inter prediction, IBC prediction, or IntraTMP prediction is performed. The model for illumination compensation can be a linear model. The image decoding device can perform illumination compensation on a sub-region to which illumination compensation is applicable among the sub-regions, and weight and combine the prediction blocks of the sub-regions to generate a final prediction block of the current block.

Alternatively, lighting compensation can be performed after generating prediction blocks of sub-regions and generating a prediction block of the current block according to a weighted sum.

For example, an image decoding device generates prediction blocks of sub-regions, and weights and combines the prediction blocks of the sub-regions to generate a prediction block of the current block. The image decoding device can use the templates of the sub-regions to generate a template corresponding to the template of the current block, and derive a model for illumination compensation using the relationship between the templates. The image decoding device can apply the derived model to the weighted prediction block of the current block, thereby generating a final prediction block with illumination compensation.

Below, the operations related to inverse transformation and restoration are described.

In relation to the inverse transformation, the entropy decoding unit (510) decodes the transform coefficients. If a secondary transformation is applied, the entropy decoding unit (510) decodes the quantized secondary transform coefficients. If a secondary transformation is not applied, the entropy decoding unit (510) decodes the quantized primary transform coefficients. The entropy decoding unit (510) parses information such as the quantization method and quantization parameter information.

The inverse quantization unit (520) inversely quantizes the decoded quantized transform coefficients based on information such as the quantization method and quantization parameter information to generate inverse quantized transform coefficients.

The inverse transform unit (530) generates residual samples by inversely transforming the TU expressed as inverse quantization transform coefficients. As shown in Fig. 6, the inverse transform unit (530) may include an inverse transform unit determination unit (610), an inverse transform kernel determination unit (612), and an inverse transform execution unit (614).

As an example, the inverse transformation unit determination unit (610) may determine a single TU or a sub-block obtained by dividing a single TU into multiple sub-blocks as the target of transformation. For example, the TU may be the entire current block, which is the target of prediction, or a portion of the current block.

As another example, if the prediction mode of the current block (i.e., prediction unit) is a geometric partitioning mode, some areas of the current block may be configured as subblock transform units (Subblock Transform Units, SbTUs) based on geometric partitioning boundary information and weighted sum areas of the current block, and the configured subblock transform units may be determined as the current transform block.

FIG. 8 is an exemplary diagram showing a sub-block transformation unit according to one embodiment of the present disclosure.

As an example, as shown in Fig. 8, among the current blocks having the size of W×H, the minimum rectangular area that includes the area that includes the weighted sum area is It is configured as a sub-block transformation unit, and the configured sub-block transformation unit can be determined as the current transformation block. The example of Fig. 8 can be applied only when the current block is predicted according to the geometric segmentation mode and the geometric segmentation boundary information can be parsed/derived and utilized. In the example of Fig. 8, A, B, C, and D are integers that are multiples of 1 or 2, and can be values having the conditions of A≤W, B≤W, C≤H, and D≤H.

FIG. 9 is an exemplary diagram showing a process for determining a sub-block transformation unit according to another embodiment of the present disclosure.

As mentioned above, as shown in Fig. 9, it can be determined based on the 1-bit flag (gsbt_flag) information.

The video decoding device checks the geometric segmentation mode flag (gpm_flag).

If the current block is not predicted according to the geometric partitioning mode (gpm_flag is false), the image decoding device determines whether the current block is transformed according to the subblock transform. Based on the sbt_flag value, the image decoding device can determine the inverse transform kernel of the subblock transform unit or the transform unit, regardless of the geometric partitioning mode.

On the other hand, if the current block is predicted according to the geometric partitioning mode (gpm_flag is true), the image decoding device checks a flag (gsbt_flag) indicating whether a geometric-based subblock transform unit ( hereinafter, used interchangeably with a subblock transform unit) is used, as shown in FIG. 9. If gsbt_flag is 1, the image decoding device can determine a subblock transform unit, as shown in FIG. 8, for example.

FIG. 10 is an exemplary diagram showing a reverse transformation process and a restoration process according to one embodiment of the present disclosure.

The video decoding device determines an inverse transform kernel with respect to the determined sub-block transform unit. After performing inverse transform based on the determined inverse transform kernel, the video decoding device can implicitly determine the positions of reconstructed residual samples based on the size of the current block, the geometric partitioning boundary of the current block, information on the weighted sum region of the current block, etc. The video decoding device can generate a reconstructed block by adding the residual samples whose positions are determined and the prediction block. Before generating the reconstructed block, the video decoding device can fill the residual values of the regions not included in the sub-block transform units within the residual block with 0, as shown in FIG. 10 (zero filling in FIG. 10).

As another example, if the prediction mode of the current block is a geometric segmentation mode, a portion of the current block may be determined as the current transformation unit based on geometric segmentation boundary information, a weighted sum region, etc. of the current block. In this case, a portion of the current block may be determined as in the following example.

FIG. 11a and FIG. 11b are exemplary diagrams showing sub-block transformation units according to another embodiment of the present disclosure.

As shown in FIGS. 11a and 11b, sub-blocks of P×Q (FIG. 11a) and R×S (FIG. 11b) can be defined based on the geometric segmentation boundary and weighted sum region information of the current block having the size of W×H. At this time, P, Q, R, and S are integers that are multiples of 1 or 2, and can be adaptively and implicitly determined based on the size of the current block, the geometric segmentation boundary of the current block, and the weighted sum region information of the current block. In FIGS. 11a and 11b, n and m can be adaptively and implicitly determined based on the size of the current block, the geometric segmentation boundary of the current block, and the weighted sum region information of the current block. According to an embodiment, the P×Q block and the R×S block can include the outside of the weighted sum region associated with the current block.

FIG. 12 is an exemplary diagram showing a reverse transformation process and a restoration process according to another embodiment of the present disclosure.

With respect to the sub-block transform unit determined as described above, the image decoding device determines an inverse transform kernel. After performing inverse transform based on the determined inverse transform kernel, the image decoding device determines P, n, Q or R, m, S values based on the size of the current block, the geometric partition boundary of the current block, information on the weighted sum region of the current block, etc., and can implicitly determine the positions of the reconstructed residual values based on the determined values. The image decoding device can generate a reconstructed block by adding the residual samples whose positions have been determined and the prediction block. Before generating the reconstructed block, the image decoding device can fill the residual values of the regions that are not included in the sub-block transform units within the residual block with 0 (filling in FIG. 12). As another example of the filling in FIG. 12, the residual values can be padded using the values of samples that are close in distance (e.g., adjacent samples) and/or the average value of samples that are close in distance.

As another example, if the prediction mode of the current block is a geometric segmentation mode, a portion of the current block may be determined as the current transformation unit based on geometric segmentation boundary information, a weighted sum region, etc. of the current block. In this case, a portion of the current block may be determined as in the following example.

FIG. 13a and FIG. 13b are exemplary diagrams showing sub-block transformation units according to another embodiment of the present disclosure.

As shown in FIGS. 13a and 13b, the number of residual samples to be inversely transformed can be derived based on geometric segmentation boundary information of the current block, weighted sum region information of the current block, size of the current block, etc. For example, all samples at the weighted sum region location can be the number of residual samples to be inversely transformed, and all samples within some sub-region including the weighted sum region within the current block can be the number of residual samples to be inversely transformed.

For example, when the number of residual samples derived as described above does not satisfy the minimum size of the transformation unit (hereinafter, minimum transformation unit), the minimum number of samples satisfying ^2K, which is larger than the derived number and larger than the minimum transformation unit, is calculated, and a sub-block transformation unit satisfying K=M×N can be defined (e.g., K= ²⁵ =32, M×N=4×8 or 8×4 in FIGS. 13A and 13B). Depending on the embodiment, the image decoding device can fill the last samples in the scanning order within M×N with the samples added to calculate K. The image decoding device can fill the last samples described above with 0. Alternatively, the image decoding device can fill the last samples with the closest sample value in the scanning order (copying), or fill them with a value generated by averaging some sample values.

The examples related to sub-block transformation units in FIGS. 13A and 13B are based on a reverse diagonal scanning order. Depending on the embodiment, other scanning orders may also be applied. FIGS. 13A and 13B are examples, and depending on the embodiment, the scanning order in the current block may be determined according to the type of prediction technique or prediction mode of each sub-region, as follows.

As an example, if both sub-regions within the current block are predicted according to inter prediction, the scanning order in the current block can be determined as one of vertical priority, horizontal priority, and zigzag, regardless of the sub-region boundary, as shown in Fig. 13a.

As another example, if both sub-regions within the current block are predicted according to IBC and/or IntraTMP, the scanning order in the current block can be determined as one of vertical priority, horizontal priority, and zigzag, regardless of the sub-region boundaries, as shown in FIG. 13a.

As another example, if at least one of the two sub-regions within the current block is intra-predicted, the scanning order in the current block may be determined by considering the sub-region boundaries, as shown in FIG. 13b. That is, samples included in one sub-region may be scanned first, and then samples included in the other sub-region may be scanned. At this time, different scanning orders may be applied to each sub-region. For example, if both sub-regions are predicted according to a directional intra-prediction mode, the scanning order may be implicitly determined based on the angle of the geometric segmentation boundary and/or the relationship between the directionality of the sub-regions.

FIG. 14 is an exemplary diagram showing a reverse transformation process and a restoration process according to one embodiment of the present disclosure.

With respect to the sub-block transform unit determined as described above, the image decoding device determines an inverse transform kernel. After performing inverse transform based on the determined inverse transform kernel, the image decoding device implicitly determines a scanning order based on the size of the current block, the geometric partitioning boundary of the current block, information on the weighted sum region of the current block, and the prediction mode of the sub-region, and can implicitly determine the positions of the reconstructed residual values according to the scanning order. The image decoding device can generate a reconstructed block by adding the residual samples whose positions have been determined and the prediction block. Before generating the reconstructed block, the image decoding device can fill the residual values of the regions that are not included in the sub-block transform units within the residual block with 0 (filling in FIG. 14). As another example of the filling in FIG. 14, the residual values can be padded using the values of samples that are close in distance (e.g., adjacent samples) and/or the average value of samples that are close in distance.

Meanwhile, the inverse transform kernel determination unit (612) can determine a separable vertical and horizontal first-order inverse transform kernel and/or a non-separable second-order inverse transform kernel, or can determine a non-separable first-order inverse transform kernel. The inverse transform performing unit (614) can inverse transform the inverse quantized transform coefficients using the determined inverse transform kernel.

For example, whether to perform a non-separable inverse transform can be determined based on the signaling/parsing of a flag or index. Here, whether to perform a non-separable inverse transform indicates whether to perform a non-separable first-order inverse transform or a non-separable second-order inverse transform. Alternatively, whether to perform a non-separable inverse transform can be implicitly determined based on the size of the current transform block. The current transform block represents a TU or sub-block transform unit.

FIG. 15 is an exemplary diagram showing a process for determining an inverse transform kernel according to one embodiment of the present disclosure.

The inverse transformation kernel decision unit (612) determines the current transformation block according to the example of Fig. 15. The type of inverse transform kernel can be determined. The order illustrated in Fig. 15 can be changed, and some orders can be omitted.

As an example, if the prediction block of the current block is predicted according to a geometric partition prediction mode, and the current transform block is a unit that restores residual signals for compensation of the aforementioned prediction block, whether to perform a non-separable inverse transform and whether to perform a non-separable first-order inverse transform with respect to the current transform block may be determined based on signaling/parsing of flags and/or indices. Alternatively, whether to perform a non-separable inverse transform and whether to perform a non-separable first-order inverse transform may be implicitly determined based on the size of the current inverse transform unit.

FIG. 16 is an exemplary diagram showing a process for determining an inverse transform kernel according to another embodiment of the present disclosure.

As another example, the inverse transform kernel decision unit (612) determines the current transform block according to the example of FIG. 16. The type of inverse transform kernel can be determined. The order illustrated in Fig. 16 can be changed, and some orders can be omitted.

As an example, whether to perform a non-separable inverse transform with respect to the current transformation block may be determined based on signaling/parsing of flags and/or indices. In some embodiments, when a non-separable inverse transform is performed, whether to perform a non-separable first inverse transform or a non-separable second inverse transform may be determined based on the size of the block. For example, when the size of the block is N×4 or 4×N (16≥N≥4) or 8×N or N×8 (16≥N≥8), a non-separable first inverse transform may be performed on the block, and a non-separable second inverse transform may be performed on blocks of different sizes.

Below, we describe a method for determining a separable first-order inverse transform kernel.

For example, if a separable first inverse transform is performed on the current transform block, the image decoding device can implicitly determine the vertical kernel and the horizontal kernel based on whether the geometric segmentation mode is performed. If the geometric segmentation mode is performed, the image decoding device can implicitly determine the vertical kernel and the horizontal kernel based on the prediction mode of each sub-region, the weighted sum region, etc. As another example, the vertical kernel and the horizontal kernel can be determined by defining a table according to an agreement between the image encoding device and the image decoding device based on the above-described information, and signaling/parsing the index of the table. At this time, different tables can be configured according to the prediction combination of the sub-regions (e.g., inter/inter prediction, inter/intra prediction, IBC/intra prediction, intra/IntraTMP prediction, etc.). Here, a combination among inter prediction, intra prediction, IBC prediction, and IntraTMP prediction can be considered as the prediction combination.

As another example, when a non-separable second-order inverse transform is performed on the current transform block, an implicitly fixed kernel (e.g., DCT2, DCT2 as vertical and horizontal kernels) can be used as the separable first-order inverse transform kernel. The fixed kernel can be adaptively changed according to the size/aspect ratio of the current transform block. When the geometric partitioning mode is performed, the fixed kernel can be adaptively changed according to the size/aspect ratio of the current transform block, the prediction mode of each sub-region, etc.

Below, we describe a method for determining a non-separable first-order inverse transform kernel.

As an example, depending on the size of the current transformation block, whether or not to perform an inseparable first-order inverse transformation can be implicitly determined.

As an example, when a non-separable first-order inverse transform is performed on the current transform block, the non-separable first-order inverse transform kernel can be determined based on the size of the current transform block and the prediction mode of each sub-region.

As another example, the non-separable first-order inverse transform can only be applied to blocks where all sub-regions are intra-predicted.

As another example, the video decoding device signals/parses an index and/or a flag indicating a non-separable first-order inverse transform kernel with respect to the current transform block. The video decoding device can determine the inverse transform kernel indicated by the parsed index/flag as the non-separable first-order inverse transform kernel of the current transform block based on the size of the current transform block, the prediction mode of each sub-region of the current block, etc. For example, when there are N types of available non-separable first-order inverse transform kernels, with respect to the transform blocks in the current block to which the geometric partitioning mode is applied, the types of available non-separable first-order inverse transform kernels may be 2 or M (M<N). As another example, with respect to the transform blocks in the current block to which the geometric partitioning mode is applied, there may be K types of available non-separable first-order inverse transform kernels, and a separate kernel set may be used.

At this time, based on the prediction combination of the sub-regions of the current block (e.g., inter/inter prediction, inter/intra prediction, IBC/intra prediction, intra/IntraTMP prediction, etc.), the types of available inverse transform kernels can be configured differently according to the agreement between the video encoding device and the video decoding device. Here, as the prediction combination, a combination among inter prediction, intra prediction, IBC prediction, and IntraTMP prediction can be considered.

As another example, if the prediction mode of each sub-region of the current block is not included in the intra prediction (such as directional intra prediction), the video decoding device may derive at least one directionality from all or part of a region of a prediction block corresponding to the current transform block, and use an inseparable first-order inverse transform kernel corresponding to one of the at least one derived directionality. Alternatively, if the prediction mode of each sub-region of the current block is included in the intra prediction (such as directional intra prediction), the video decoding device may derive at least one directionality from all or part of a region of a prediction block corresponding to the current transform block, and use an inseparable first-order inverse transform kernel corresponding to one of the at least one derived directionality. For example, a directionality similar to the intra prediction mode of each sub-region of the current block among the derived directionality may be used. The video decoding device applies a Sobel filter to an area corresponding to a current transformation block in a prediction block (in the case of a sub-block transformation unit, see examples of FIGS. 8, 11a and 11b, 13a and 13b) to calculate vertical/horizontal gradient and size values for each preset unit. The video decoding device can construct a Histogram of Gradient (HoG) from the vertical/horizontal gradient and size values for each preset unit, and derive directionality based on the HoG. Here, the Sobel filter represents an edge detection filter, and the preset unit may be, for example, a pixel or a pixel group.

As another example, with respect to the current transformation block, if an inseparable second-order inverse transform is used, the use of an inseparable first-order inverse transform is restricted. In some embodiments, if the inseparable first-order inverse transform index is parsed first, whether or not the inseparable second-order inverse transform is parsed may depend on the index value of the inseparable first-order inverse transform.

Below, we describe a method for determining a non-separable second-order inverse transform kernel.

As an example, depending on the size of the current transformation block, whether or not to perform an inseparable second-order inverse transformation can be implicitly determined.

As an example, when a non-separable second-order inverse transform is performed on the current transform block, the non-separable second-order inverse transform kernel can be determined based on the size of the current transform block and the prediction mode of each sub-region.

As another example, the non-separable second-order inverse transform can only be applied to blocks where all sub-regions are intra-predicted.

As another example, the video decoding device signals/parses an index and/or a flag indicating a non-separable second-order inverse transform kernel with respect to the current transform block. The video decoding device can determine the inverse transform kernel indicated by the parsed index/flag as the non-separable second-order inverse transform kernel of the current transform block based on the size of the current transform block, the prediction mode of each sub-region of the current block, etc. For example, when there are N types of available non-separable second-order inverse transform kernels, with respect to the transform blocks in the current block to which the geometric partitioning mode is applied, the types of available non-separable second-order inverse transform kernels may be 2 or M (M<N). As another example, with respect to the transform blocks in the current block to which the geometric partitioning mode is applied, there may be K types of available non-separable second-order inverse transform kernels, and a separate kernel set may be used.

At this time, based on the prediction combination of the sub-regions of the current block (e.g., inter/inter prediction, inter/intra prediction, IBC/intra prediction, intra/IntraTMP prediction, etc.), the types of available inverse transform kernels can be configured differently according to the agreement between the video encoding device and the video decoding device. Here, as the prediction combination, a combination among inter prediction, intra prediction, IBC prediction, and IntraTMP prediction can be considered.

As another example, if the prediction mode of each sub-region of the current block is not included in the intra prediction (such as directional intra prediction), the video decoding device may derive at least one directionality from all or part of a region of a prediction block corresponding to the current transform block, and use an inseparable second-order inverse transform kernel corresponding to one of the at least one derived directionality. Alternatively, if the prediction mode of each sub-region of the current block is included in the intra prediction (such as directional intra prediction), the video decoding device may derive at least one directionality from all or part of a region of a prediction block corresponding to the current transform block, and use an inseparable second-order inverse transform kernel corresponding to one of the at least one derived directionality. For example, a directionality similar to the intra prediction mode of each sub-region of the current block among the derived directionality may be used. The video decoding device applies a Sobel filter to an area corresponding to the current transformation block in the prediction block (in the case of a sub-block transformation unit, see examples of FIGS. 8, 11a and 11b, 13a and 13b) to calculate vertical/horizontal slope and size values for each preset unit. The video decoding device can configure a HoG from the vertical/horizontal slope and size values for each preset unit, and derive directionality based on the HoG. Here, the Sobel filter represents a boundary detection filter, and the preset unit may be, for example, a pixel or a pixel group.

As another example, in relation to the current transform block, when a non-separable second inverse transform is performed, the first inverse transform may be performed using separable vertical and horizontal inverse transform kernels. As the kernel used for the first inverse transform, a fixed kernel (e.g., DCT2, DCT2 as a vertical and horizontal kernel) may be used according to an agreement between the image encoding device and the image decoding device based on the type of the second inverse transform kernel.

Hereinafter, a method for transforming and inversely transforming a transform block of a current block is described. The examples of FIGS. 17 and 18 are described based on the case where the current transform block is a sub-block transform unit, but can be similarly applied when the current transform block is a TU.

FIG. 17 is a flowchart illustrating a method by which an image encoding device transforms a transform block according to one embodiment of the present disclosure.

The video encoding device obtains residual samples of the current block (S1700).

If the current block is predicted according to the geometric partitioning mode, the image encoding device obtains a flag indicating whether a sub-block transform unit is used from a higher level (S1702).

The video encoding device checks whether the flag is true (S1704).

If the flag indicates the use of a sub-block transform unit as true (Yes in S1704), the video encoding device may perform the following steps.

The video encoding device determines a prediction region used to configure a sub-block transformation unit of the current block (S1706).

Here, the prediction region may include a weighted sum region of sub-regions, with respect to sub-regions of the current block divided according to a geometric division mode. For example, the image encoding device may determine the prediction region based on a minimum rectangular region among rectangular regions including the weighted sum region of the sub-regions.

For example, if the size of the prediction region does not satisfy the minimum size of the transformation unit, the video encoding device can calculate the minimum number of samples that satisfy 2 ^K (where K is a natural number). Here, 2 ^K is larger than the prediction region and larger than the minimum transformation unit. The video encoding device can determine the size of the sub-block transformation unit based on the minimum number of samples.

The video encoding device fills the sub-block transformation unit according to a predetermined scanning order using residual samples of the prediction region among the residual samples of the current block (S1708).

When a sub-block transform unit is larger than a prediction region, the video encoding device may pad the last positions of the sub-block transform unit according to a predetermined scanning order with zero samples, the closest sample value in the scanning order, or a value generated by averaging some samples.

The video encoding device determines a transformation kernel of a subblock transformation unit (S1710).

An image encoding device can obtain non-separable transform information from a higher level. Here, the non-separable transform information indicates whether a non-separable transform is applied, and the non-separable transform may indicate a non-separable primary transform or a non-separable secondary transform. The non-separable transform information may be a flag or index indicating whether a non-separable transform is applied.

As an example, when the non-separable transform information instructs the performance of a non-separable transform, the image encoding device can determine a non-separable transform kernel corresponding to a non-separable first transform kernel or a non-separable second transform kernel based on the size of the sub-block transform unit and the prediction modes of the sub-regions of the current block.

As another example, when the non-separable transform information indicates the performance of a non-separable transform, the video encoding device can determine whether to perform a non-separable primary transform or a non-separable secondary transform based on the size of the sub-block transform unit.

The video encoding device applies a transform kernel to a sub-block transform unit to generate transform coefficients of the sub-block transform unit (S1712).

Thereafter, the video encoding device can encode the transform coefficients of the flag and sub-block transform units.

On the other hand, if the flag is false and the sub-block transform unit is not used (No in S1704), the video encoding device can determine the transform kernel of the current block and apply the determined transform kernel to the residual samples of the current block to generate transform coefficients of the current block. Thereafter, the video encoding device can encode the flag and the transform coefficients of the current block.

FIG. 18 is a flowchart illustrating a method for an image decoding device to inversely transform a transform block according to one embodiment of the present disclosure.

The image decoding device obtains the inverse quantized transform coefficients of the current block (S1800).

If the current block is predicted according to the geometric partitioning mode, the image decoding device decodes a flag indicating whether a sub-block transform unit is used from the bitstream (S1800).

The video decoding device checks whether the flag is true (S1804).

If the flag indicates the use of a sub-block transform unit as true (Yes in S1804), the video decoding device may perform the following steps.

The video decoding device determines the sub-block transformation unit of the current block (S1806).

Here, the sub-block transformation unit may be determined based on a prediction region including a weighted sum region of the sub-regions of the current block divided according to a geometric division mode. For example, the image decoding device may determine the sub-block transformation unit based on a minimum rectangular region among rectangular regions including the weighted sum region of the sub-regions.

The video decoding device determines an inverse transform kernel of a sub-block transform unit (S1808).

A video decoding device decodes non-separable transform information from a bitstream. The non-separable transform information indicates whether a non-separable inverse transform is applied, and the non-separable inverse transform may represent a non-separable first-order inverse transform or a non-separable second-order inverse transform. The non-separable transform information may be a flag or index indicating whether a non-separable transform is applied.

As an example, when the non-separable transform information instructs the performance of a non-separable inverse transform, the image decoding device can determine a non-separable inverse transform kernel corresponding to a non-separable first inverse transform kernel or a non-separable second inverse transform kernel based on the size of the sub-block transform unit and the prediction modes of the sub-regions of the current block.

As another example, when the non-separable transform information indicates the performance of a non-separable inverse transform, the image decoding device can determine whether to perform a non-separable first inverse transform or a non-separable second inverse transform based on the size of the sub-block transform unit.

As another example, when performing a non-separable first inverse transform based on the size of a sub-block transform unit, the image decoding device can decode an index from a bitstream. Here, the index can indicate one of the non-separable first inverse transform kernels included in a table, with respect to a table including non-separable first inverse transform kernels defined based on a predicted combination of sub-regions. The image decoding device can obtain the non-separable first inverse transform kernel of the sub-block transform unit from the table based on the index.

As another example, when a non-separable first inverse transform is performed and the prediction mode of each sub-region of the current block is not a directional prediction mode, the image decoding device can generate a HoG based on a prediction region corresponding to a prediction block or a sub-block transformation unit of the current block, and derive at least one directional prediction mode corresponding to a plurality of directionality that has accumulated a lot based on the HoG. When a non-separable first inverse transform is performed and the prediction mode of each sub-region of the current block is a directional prediction mode, the image decoding device can generate a HoG based on a prediction region corresponding to a prediction block or a sub-block transformation unit of the current block, and derive at least one directional prediction mode corresponding to a plurality of directionality that has accumulated a lot from the HoG. The image decoding device can determine a non-separable first inverse transform kernel of the sub-block transformation unit based on one of the at least one derived directional prediction mode.

On the other hand, when the non-separable first inverse transform is not performed, the image decoding device can decode an index from the bitstream. Here, the index can indicate one of the first inverse transform kernel pairs included in the table, with respect to a table including pairs of first inverse transform kernels in vertical and horizontal directions defined based on a predicted combination and a weighted sum region of sub-regions. The image decoding device can obtain the first inverse transform kernel pairs in the vertical and horizontal directions of the sub-block transform unit from the table based on the index.

As an example, when performing a non-separable second inverse transform based on the size of a sub-block transform unit, the image decoding device can decode an index from a bitstream. Here, the index can indicate one of the non-separable second inverse transform kernels included in a table, with respect to a table including non-separable second inverse transform kernels defined based on a predicted combination of sub-regions. The image decoding device can obtain the non-separable second inverse transform kernel of the sub-block transform unit from the table based on the index.

As another example, when performing a non-separable second-order inverse transform and the prediction mode of each sub-region of the current block is not a directional prediction mode, the image decoding device may perform a prediction region based on the prediction region corresponding to the prediction block or sub-block transform unit of the current block. A HoG can be generated and at least one directional prediction mode corresponding to a large number of accumulated directions from the HoG can be derived. When a non-separable second inverse transformation is performed and the prediction mode of each sub-region of the current block is a directional prediction mode, the image decoding device can decode the prediction region corresponding to the prediction block or sub-block transformation unit of the current block based on the prediction region. A HoG can be generated, and at least one directional prediction mode corresponding to a large number of accumulated directions can be derived from the HoG. The image decoding device can determine a non-separable second-order inverse transform kernel of a sub-block transform unit based on at least one of the derived directional prediction modes.

The image decoding device applies an inverse transform kernel to inverse quantized transform coefficients to generate residual samples of a sub-block transform unit (S1810).

The video decoding device fills the prediction area used to determine the sub-block transformation unit according to a predetermined scanning order using residual samples of the sub-block transformation unit (S1812).

If the sub-block transform unit is larger than the prediction region, the image decoding device can adjust the size of the sub-block transform unit to be the same as the size of the prediction region by discarding the last residual samples according to the scanning order determined from the sub-block transform unit.

The image decoding device generates a residual block of the current block by padding zero samples around the prediction region (S1814).

On the other hand, if the flag is false and the sub-block transform unit is not used (No of S1804), the image decoding device can determine the inverse transform kernel of the current block and apply the determined inverse transform kernel to the inverse quantized transform coefficients of the current block to generate the residual block of the current block.

Although the flowchart/timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart/timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart/timing diagram is not limited to a chronological order.

It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.

Meanwhile, the various functions or methods described in this embodiment may also be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

The above description is merely an example of the technical idea of the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment, but rather to explain it, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application claims priority to Korean patent application No. 10-2024-0050270, filed in Korea on April 15, 2024, and Korean patent application No. 10-2025-0038023, filed in Korea on March 25, 2025, the entire contents of which are incorporated herein by reference.

Claims (17)

In a method for restoring a current block, performed by a video decryption device, A step of obtaining the inverse quantized transform coefficients of the current block; and A step of decoding a flag indicating whether a sub-block transform unit is used from the bitstream when the current block is predicted according to a geometric partitioning mode. Including, If the above flag indicates the use of the above subblock transformation unit, A step of determining a sub-block transformation unit of the current block, wherein the sub-block transformation unit is determined based on a prediction region including a weighted sum region of sub-regions of the current block divided according to a geometric division mode; A step of determining an inverse transform kernel of the above sub-block transform unit; A step of generating residual samples of the sub-block transform unit by applying the inverse transform kernel to the inverse quantized transform coefficients; and Using the residual samples of the above sub-block transformation unit, according to the predetermined scanning order A step of filling the prediction area used to determine the above sub-block transformation unit. A method further comprising: In the first paragraph, The step of determining the above sub-block conversion unit is: A method for determining the sub-block transformation unit based on the minimum rectangular area among rectangular areas including the weighted sum area of the sub-areas. In the first paragraph, The step of filling the above prediction area is: A method for adjusting the size of the sub-block transformation unit to be the same as the size of the prediction area by discarding the last residual samples according to the predetermined scanning order from the sub-block transformation unit when the sub-block transformation unit is larger than the prediction area. In the first paragraph, The step of determining the above inverse transform kernel is: A step of decoding non-separable transform information from the bitstream, The above non-separable transformation information indicates whether a non-separable inverse transformation is applied, and the non-separable inverse transformation represents a non-separable first-order inverse transformation or a non-separable second-order inverse transformation. If the above non-separable transformation information instructs the performance of the above non-separable inverse transformation, A step of determining a non-separable inverse transform kernel corresponding to a non-separable first inverse transform kernel or a non-separable second inverse transform kernel based on the size of the sub-block transform unit and the prediction modes of the sub-regions of the current block. A method further comprising: In paragraph 4, The step of determining the above inverse transform kernel is: If the above non-separable transformation information instructs the performance of the above non-separable inverse transformation, A method further comprising a step of determining whether to perform the non-separable first-order inverse transform or the non-separable second-order inverse transform based on the size of the sub-block transform unit. In paragraph 5, The step of determining the above inverse transform kernel is: When performing the non-separable first-order inverse transformation based on the size of the above sub-block transformation unit, A step of decoding an index from the bitstream, wherein the index points to one of the non-separable first-order inverse transform kernels included in a table, the non-separable first-order inverse transform kernels being defined based on a predicted combination of the sub-regions; and A step of obtaining a non-separable first-order inverse transform kernel of the sub-block transform unit from the table based on the index. A method further comprising: In paragraph 5, The step of determining the above inverse transform kernel is: If the above non-separable first-order inverse transformation is performed and the prediction mode of each sub-region of the current block is not a directional prediction mode, A step of generating a Histogram of Oriented Gradient (HoG) based on the above prediction area and deriving at least one directional prediction mode corresponding to a large number of accumulated directions from the HoG; and A step of determining a non-separable first-order inverse transform kernel of the sub-block transform unit based on at least one of the above-described directional prediction modes. A method further comprising: In paragraph 5, The step of determining the above inverse transform kernel is: If the above non-separable first-order inverse transformation is not performed, A step of decoding an index from the bitstream, wherein the index points to one of the pairs of first-order inverse transform kernels included in a table including pairs of first-order inverse transform kernels in vertical and horizontal directions defined based on the predicted combination of the sub-regions and the weighted sum region; and A step of obtaining a pair of vertical and horizontal first-order inverse transform kernels of the sub-block transform unit from the table based on the index. A method further comprising: In paragraph 5, The step of determining the above inverse transform kernel is: When performing the non-separable second-order inverse transformation based on the size of the above sub-block transformation unit, A step of decoding an index from the bitstream, wherein the index points to one of the non-separable second-order inverse transform kernels included in a table, the non-separable second-order inverse transform kernels being defined based on a predicted combination of the sub-regions; and A step of obtaining a non-separable second-order inverse transform kernel of the sub-block transform unit from the table based on the index. A method further comprising: In paragraph 5, The step of determining the above inverse transform kernel is: If the above non-separable second inverse transformation is performed and the prediction mode of each sub-region of the current block is not a directional prediction mode, A step of generating a Histogram of Oriented Gradient (HoG) based on the above prediction area, and deriving at least one directional prediction mode corresponding to a large number of accumulated directions based on the HoG; and A step of determining a non-separable second-order inverse transform kernel of the sub-block transform unit based on at least one of the above-described directional prediction modes. A method further comprising: In the first paragraph, If the above flag indicates the use of the above subblock transformation unit, A step of generating a residual block of the current block by padding zero samples around the prediction region. A method further comprising: In a method for encoding a current block performed by a video encoding device, A step of obtaining residual samples of the current block; and A step of obtaining a flag indicating whether a sub-block transformation unit is used from a higher level when the current block is predicted according to a geometric partitioning mode. Including, If the above flag indicates the use of the above subblock transformation unit, A step of determining a prediction region used to form a sub-block transformation unit of the current block, wherein the prediction region includes a weighted sum region of sub-regions of the current block divided according to a geometric division mode; A step of filling the sub-block transformation unit according to a predetermined scanning order using residual samples of the prediction region among residual samples of the current block; a step of determining a transformation kernel of the above sub-block transformation unit; and A step of generating transformation coefficients of the sub-block transformation unit by applying the transformation kernel to the sub-block transformation unit. How to include more. In paragraph 12, The step of determining the above prediction area is: A method for determining the prediction region based on the minimum rectangular region among rectangular regions including the weighted sum region of the above sub-regions. In paragraph 12, If the size of the above prediction region does not satisfy the minimum size of the transformation unit, A step of calculating the minimum number of samples satisfying 2 ^K (where K is a natural number), wherein 2 ^K is larger than the prediction region and larger than the minimum transformation unit; and A step of determining the size of the sub-block transformation unit based on the number of the minimum samples. A method further comprising: In Article 14, The step of filling the above sub-block conversion unit is: A method of padding zero samples to the last positions of the subblock transformation unit according to the predetermined scanning order when the subblock transformation unit is larger than the prediction region. In paragraph 12, The step of determining the above transformation kernel is: comprising a step of obtaining non-separable transform information from the upper level; The above non-separable transformation information indicates whether a non-separable transformation is applied, and the non-separable transformation indicates a non-separable first transformation or a non-separable second transformation. If the above non-separable conversion information instructs the performance of the above non-separable conversion, A step of determining a non-separable transform kernel corresponding to a non-separable first transform kernel or a non-separable second transform kernel based on the size of the sub-block transform unit and the prediction modes of the sub-regions of the current block. A method further comprising: In a method for providing video data to a video decoding device, A step of encoding the above video data into a bitstream; and A step of transmitting the above bitstream to the image decoding device Including, The step of encoding the above video data is: A step of obtaining residual samples of the current block; and A step of obtaining a flag indicating whether a sub-block transformation unit is used from a higher level when the current block is predicted according to a geometric partitioning mode. Including, If the above flag indicates the use of the above subblock transformation unit, A step of determining a prediction region used to form a sub-block transformation unit of the current block, wherein the prediction region includes a weighted sum region of sub-regions of the current block divided according to a geometric division mode; A step of filling the sub-block transformation unit according to a predetermined scanning order using residual samples of the prediction region among residual samples of the current block; a step of determining a transformation kernel of the above sub-block transformation unit; and A step of generating transformation coefficients of the sub-block transformation unit by applying the transformation kernel to the sub-block transformation unit. A method further comprising: