RU2816350C2 - Limitations of cross-component mode applicability - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to video encoding and decoding. Technical result is provided due to the fact that performing the conversion between the current video block of the chromatic component of visual media data and the representation of the current video block of the chromatic component in the form of a data bit stream, wherein in the conversion process, the chromaticity component residual of the current chroma component video block is scaled based on the scaling factor, wherein the scaling factor is determined based at least on the readings of the brightness component located in predetermined positions.

EFFECT: high efficiency of encoding and decoding video.

14 cl, 23 dwg

Description

Field of technology to which the invention relates

This document relates to technologies, systems and devices for video encoding and decoding.

BACKGROUND OF THE INVENTION

Despite advances in video compression, digital video still takes up the largest share of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and presenting video on a display increases, bandwidth requirements for digital video use are expected to continue to increase.

Disclosure of the invention

Devices, systems and methods are provided with respect to digital video encoding/decoding and, in particular, with respect to obtaining a simplified linear model for a cross-component linear model (CCLM) prediction mode in the field of video encoding/decoding. The techniques described here can be applied to both existing video coding standards (such as High Efficiency Video Coding (HEVC)) and future video coding standards (such as Versatile Video Coding). VVC))) or corresponding codecs.

According to one representative aspect, a method for processing visual media data is provided. The method comprises performing a conversion between a current chrominance video block of the visual media and a representation of the current chrominance video block as a stream of data bits, wherein during the conversion process, the remainder of the chrominance component for the current chrominance video block is scaled based on a scaling factor, wherein the scaling factor is obtained at least based on the luminance component samples located at given positions.

According to one representative aspect, a method for processing visual media signals is provided. The method comprises the step of performing a conversion between a current video block of visual media data and a representation of the current video block as a stream of data bits, wherein the conversion process determines a second set of color component values for the current video block using a cross-component linear model (CCLM) and/or step processing in luma mapping with chroma scaling (LMCS) mode from the first set of values of the color components of visual media data.

According to another representational aspect, a method for processing visual media data is provided. The method comprises the steps of performing a conversion between a current video block of visual media data and a representation of the current video block as a stream of data bits, wherein during the conversion process, one or more reconstructed samples associated with the current frame of visual media data are used to obtain a scaling factor of the remaining chrominance component in the step processing in the luminance component display mode with chrominance component scaling (LMCS).

According to another representational aspect, a method for processing visual media data is provided. The method comprises the steps of checking, during the conversion process between the current chroma video block and the representation of the current chroma video block as a stream of data bits, the availability of one or more adjacent luminance blocks relative to the corresponding luminance block that covers the upper left sample located in that the same location of the brightness component block; determining, based on the availability of one or more adjacent luminance component blocks, whether adjacent luminance component samples should be extracted from said corresponding luminance component block; obtain, based on said definition, a scaling factor; scaling, based on the specified scaling factor, the chrominance component residual of the current chrominance component video block to generate a scaled chroma component residual; and performing a transformation based on the scaled chrominance component residual.

According to another representational aspect, a method for processing visual media data is provided. The method comprises obtaining, in a conversion process between the current video block of visual media data and a representation of the current video block as a stream of data bits, a second set of color component values for the current video block from the first set of color component values of the visual media data using a model associated with the processing step data, wherein the first set of color component values are adjacent samples relative to a corresponding luminance component block covering the upper left sample of a co-located luminance component block.

According to another representational aspect, a method for processing visual media data is provided. The method comprises, in the process of converting between a current chrominance video block of visual media data and a representation of the current chrominance video block as a stream of data bits, a step in which a determination is made regarding selective activation or deactivation of the application of a cross-component linear model (CCLM) and/or scaling of the remaining chrominance component (chroma residual scaling (CRS)) to the current chroma video block based at least in part on one or more conditions associated with a colocated luminance block of the current chroma video block.

According to another representational aspect, a method for processing visual media data is provided. The method comprises selectively activating or deactivating the application of luminance-dependent scaling of a chrominance residual (CRS) to chrominance components of a current video block of visual media to encode the current video block in a video domain of the visual media into a representation of the visual media as a stream of data bits; and making a determination regarding the inclusion and exclusion of a field in the representation of the visual media data as a stream of data bits, the field indicating selective activation or deactivation, and, if the field is included, being signaled at a layer different from the first syntactic layer associated with the current video block.

According to another representational aspect, a method for processing visual media data is provided. The method comprises parsing a field in a representation of visual media data as a stream of data bits, the field being included at a level different from a first syntax level associated with the current video block; and selectively activating or deactivating, based on the field, applying luminance-dependent scaling of a chrominance residual (CRS) to color components of the current video block of visual media to generate a decoded video region based on the data bitstream representation.

According to another representational aspect, a method for processing visual media data is provided. The method comprises selectively activating or deactivating the application of a cross-component linear model (CCLM) to a current video block of visual media to encode the current video block into a representation of the visual media as a stream of data bits; and making a determination regarding the inclusion or exclusion of a field in the representation of the visual media data as a bit stream of data, the field indicating selective activation or deactivation and, if the field is included, being signaled at a layer different from the first syntactic layer associated with the current video block.

According to another representational aspect, a method for processing visual media data is provided. The method comprises parsing a field in said representation of visual media data as a stream of data bits, the field being included at a level different from a first syntax level associated with the current video block; and selectively activating or deactivating, based on the specified field, applying a cross-component linear model (CCLM) to the current video block of visual media to generate a decoded video region based on the data bitstream representation.

According to yet another exemplary aspect, there is provided a video encoding or decoding device comprising a processor configured to implement the method described above.

According to another exemplary aspect, a computer-readable software medium is provided. The medium stores code containing processor-executable instructions to perform one of the disclosed methods.

In yet another exemplary aspect, the method described above is implemented as processor-executable code and stored on a computer-readable storage medium.

According to yet another representative aspect, a device is provided that is capable of performing said method. The device may include a processor programmed to perform the method.

The above and other aspects and features of the proposed technology are described in more detail in the drawings, description and claims.

Brief description of drawings

Fig. 1 shows an example of intra-prediction angular modes in the HEVC coding standard.

Fig. 2 shows an example of directional modes not in the HEVC encoding standard.

Fig. 3 shows an example in conjunction with a mode based on the CCLM model.

Fig. 4 shows an example of a luma display architecture with chrominance scaling.

Fig. 5 shows an example of a luma block and a chrominance block in different color formats.

Fig. 6 shows an example of a luminance component block and a chrominance component block in the same color format.

Fig. 7 shows an example of a colocated luma block covering multiple formats.

Fig. 8 shows an example of a luminance component block within a larger luminance component block.

Fig. 9 shows an example of a luma block within a larger luma block and within a bounding box.

Fig. 10 is a block diagram of an example hardware platform for implementing the methods for decoding visual media data or encoding visual media data described herein.

Fig. 11 shows a logic diagram of an example method for generating a linear model for cross-component forecasting in accordance with the proposed technology.

Fig. 12 is a block diagram of an example video processing system in which the proposed technology may be implemented.

Fig. 13 shows a logic diagram of an example of a method for processing visual media data.

Fig. 14 shows a logic diagram of an example of a method for processing visual media data.

Fig. 15 shows a logic diagram of an example of a method for processing visual media data.

Fig. 16 shows a logic diagram of an example of a method for processing visual media data.

Fig. 17 shows a logic diagram of an example of a method for processing visual media data.

Fig. 18 shows a logic diagram of an example of a method for processing visual media data.

Fig. 19 shows a logic diagram of an example of a method for processing visual media data.

Fig. 20 shows a logic diagram of an example of a method for processing visual media data.

Fig. 21 shows a logic diagram of an example of a method for processing visual media data.

Fig. 22 shows a logic diagram of an example of a method for processing visual media data.

Fig. 23 shows a logic diagram of an example of a method for processing visual media data.

Carrying out the invention

2.1 Brief overview of the HEVC encoding standard

2.1.1. Intra-frame prediction mode in HEVC/H.265 standard

Intra-frame prediction involves generating samples for the specific transform block under consideration (TB (transform block)) using samples previously reconstructed in the color channel under consideration. The intra prediction mode is signaled separately for the luminance and chroma channels, so that the intra prediction mode for the chroma channel can optionally depend on the intra prediction mode for the luminance channel via the 'DM_CHROMA' mode. Although the intra-prediction mode is signaled at the PB (prediction block) level, the intra-prediction procedure is applied at the TB level, in accordance with the square tree hierarchy for the remainder for the CU in question, thereby allowing one TB to be encoded affecting the encoding of the next TB block in the same CU under consideration, and therefore reducing the distance to the samples used as reference values.

HEVC coding uses 35 intra-prediction modes—DC mode, planar mode, and 33 directional or “angular” intra-prediction modes. These 33 angular intra-prediction modes are illustrated in FIG. 1.

For PB blocks associated with chrominance color channels, the intra-frame prediction mode is specified as either planar mode, DC mode, horizontal mode, vertical mode, 'DM_CHROMA' mode, or sometimes '34' diagonal mode.

Note that for 4:2:2 and 4:2:0 chrominance formats, a chrominance component PB block may be superimposed on two or four (respectively) PB blocks; in this case, the luminance component direction for the DM_CHROMA mode is taken from the upper left part of these luminance component blocks PB.

The DM_CHROMA mode indicates that the intra-prediction mode for the luminance component color channel block PB is applied to the chrominance component color channel blocks PB. Because this is relatively general, the most likely mode encoding scheme for intra_chroma_pred_mode is biased in favor of this selected mode.

2.2 Description of the Versatile Video Coding (VVC) algorithm

- VVC encoding architecture

For research into future video encoding technologies that will be developed after HEVC encoding technology, the Video Coding Expert Group (VCEG) (Q6/16) and the ISO/IEC JTC1 SC29/WG11 (MPEG) Division jointly founded a Joint Group in 2015 research in the field of video (Joint Video Exploration Team (JVET)). The JVET meets by consensus quarterly, and the new encoding standard aims to achieve a 50% reduction in the required data bit rate compared to HEVC encoding. The new video encoding standard was officially named Versatile Video Coding (VVC) at the April 2018 JVET meeting, and the first version of the VVC encoding test model (VTM models) was released at that time. As VVC encoding standardization efforts continue continuously, more and more new encoding methods are added to the VVC encoding standard at each JVET meeting. Then the working draft of the VVC coding standard and the VTM test model are updated after each meeting. The VVC coding project is currently targeting technical completion (FDIS) at the July 2020 meeting.

Like most previous standards, the VVC coding standard has a hybrid block-based coding architecture combining inter- and intra-frame prediction coding and entropy transform coding. The image partitioning structure splits the input video into blocks called coding tree units (CTUs). The CTU is partitioned using a square tree with a nested multi-type tree structure into coding units (CUs), where the leaf (“leaf” of the tree) coding unit (CU) defines the region sharing the same prediction mode ( for example, intra-frame or inter-frame prediction). As used herein, the term 'unit' defines the area of the image that covers all color components; The term 'block' is used to define the area covering a particular color component (eg luminance component), and may differ in spatial arrangement when the chroma sample format, such as 4:2:0, is taken into account.

2.2.2 Double/split tree partitioning in the VVC coding standard

The luminance component and the chrominance component may have separate partition trees for I-slices. Split tree partitioning is done at the 64x64 block level instead of the CTU level. The VTM model software has an SPS set flag to control whether the dual tree structure is enabled or disabled.

2.2.3 Intra-frame prediction in the VVC coding standard

2.2.3.1 67 intra-frame prediction modes

To capture arbitrary edge directions present in natural video, the number of intra-prediction directional modes in the VTM4 model is expanded from 33, as used in the HEVC encoding standard, to 65. New directional modes not present in the HEVC encoding standard are depicted by the red dotted arrows in FIG. . 2, and planar mode and DC mode remain the same. These distributed, more densely targeted intra-prediction modes apply to intra-prediction for all block sizes and both luma and chrominance components.

2.2.3.2 Forecasting using Cross Component Linear Model (CCLM)

To reduce cross-component redundancy, the VTM4 model uses a cross-component linear model (CCLM) prediction mode in which chrominance samples are predicted from reconstructed luminance samples from the same CU using a linear model as follows:

Where represents the predicted chrominance samples in some CU unit and represents subsampled reconstructed luminance samples from the same CU. Options And linear model is determined from the relationship between the luminance values and chrominance values from two samples, which are the luminance component samples with the minimum sample value and the maximum luminance component sample value within a set of adjacent samples of the subsampled luminance component, and the corresponding chrominance component samples. Options And linear model is obtained according to the following equations.

where Y _a and X _a represent a luminance value and a chrominance value corresponding to a luminance component sample with a maximum luminance component sample value. Both X _b and Y _b represent a luminance value and a chrominance value corresponding to a luminance component sample with a minimum luminance component sample value. Fig. 3 shows an example of the left and top sample positions and the current block sample used in the CCLM model-based mode.

The partitioning operation to calculate the parameter α is implemented using a conversion table. To reduce the amount of memory required to store the table, the value of the diff parameter (the difference between the maximum and minimum values) and the α parameter are expressed in exponential form. For example, the diff parameter is approximated by a 4-bit significant part and an exponential part. Therefore, the table for the value 1/diff is reduced to 16 elements for 16 mantissa values as follows:

DivTable [ ] = {0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0}

This should provide benefits both in reducing the complexity of calculations and in reducing the amount of memory required to store the necessary tables.

In addition to using the top template and the left template together to calculate linear model coefficients, they can be used alternatively in two other LM modes, called LM_A and LM_L modes.

In LM_A mode, only the template on top is used to calculate the coefficients of the linear model. To obtain a larger number of samples, the template from above is expanded to (W+H). In LM_L mode, only the template on the left is used to calculate the coefficients of the linear model. To obtain more samples, the pattern on the left is expanded to (H+W).

For a non-square block, the pattern on the top is expanded to W+W, and the pattern on the left is expanded to H+H.

To match the positions of the chrominance component samples for video sequences in 4:2:0 format, subsampling filters of two types are applied to the luminance component samples to achieve a subsampling ratio (resolution reduction) of 2 to 1 in both horizontal and vertical directions. The selection of the downsampling filter is specified by a flag at the SPS set level. These two downsampling filters are described by the following relationships, which correspond to the content “type-0” and “type-2”.

Note that only one luminance row (generalized intra-prediction row buffer) is used to generate subsampled luminance samples when the top reference row is located at the CTU boundary.

This parameter calculation is carried out as part of the decoding procedure, and is not simply a search operation in the encoder. As a result, no syntax is used to pass the values of the α and β parameters to the decoder.

To encode the color component using intra-frame prediction, a total of 8 intra-frame prediction modes can be used. The set of these eight modes contains five traditional intra-frame prediction modes and three cross-component linear model modes (CCLM, LM_A and LM_L). The coding mode of the chrominance component directly depends on the intra-frame prediction mode of the corresponding block of the luminance component. Since I-slices can use separate block partitioning structures for luma and chrominance components, one chroma block can correspond to multiple luma blocks. Therefore, for the DM chroma coding mode (Chroma DM mode), the intra-prediction mode for the corresponding luminance block covering the central position of the current chrominance block is directly “inherited”. 2.2.3.2.1 Corresponding modified detailed design (JVET-N0271)

The following description is based on a modified detailed design in JVET-M1001 and its application in JVET-N0271. The modification to the accepted document JVET-N0220 is shown in bold with underlining.

Syntax table

RBSP Sequence Parameter Set Syntax

Flag , equal to 0 specifies that intra-frame prediction using a cross-component linear model from luma to chrominance is not enabled. The sps_cclm_enabled_flag flag equal to 1 specifies that intra-frame prediction using a cross-component linear model from luma to chrominance is enabled.

The predicted samples predSamples[ x ][ y ] for x = 0..nTbW − 1, y = 0..nTbH − 1 are determined as follows:

- If both numSampL and numSampT are 0, the following applies:

predSamples[ x ][ y ] = 1 << ( BitDepth _C − 1 ) (8-162)

- Otherwise, the following ordered steps apply:

1. The brightness component samples located in the same place pY[ x ][ y ] at x = 0..nTbW * 2 − 1, y= 0..nTbH * 2 – 1 are set equal to the reconstructed brightness component samples before the deblocking filtering procedure in positions ( xTbY + x, yTbY + y ).

2. Adjacent samples pY[ x ][ y ] of the brightness component are determined as follows:

- When the number numSampL is greater than 0, the adjacent left samples pY[ x ][ y ] of the luminance component at x = −1..−3, y = 0..2 * numSampL − 1 are set equal to the reconstructed samples of the luminance component before the deblocking filtering procedure at positions ( xTbY + x , yTbY +y ).

- When the number numSampT is greater than 0, the upper adjacent samples pY[ x ][ y ] of the brightness component at x = 0..2 * numSampT − 1, y = −1, −2, are set equal to the reconstructed samples of the brightness component before the deblocking filtering procedure in positions ( xTbY+ x, yTbY + y ).

- When the variable availTL is equal to “True” (TRUE), the adjacent upper left luminance samples pY[ x ][ y ] at x = −1, y = −1, −2 are set equal to the reconstructed luminance samples before the deblocking filtering procedure in positions ( xTbY+ x, yTbY + y ).

3. Subsampled samples pDsY[ x ][ y ] of the brightness component located in the same place at x = 0..nTbW − 1, y = 0..nTbH − 1 are determined as follows:

- If sps_cclm_colocated_chroma_flag is 1, the following applies:

- pDsY[ x ][ y ] for x = 1..nTbW − 1, y = 1..nTbH − 1 is determined as follows:

pDsY[ x ][ y ] = ( pY[ 2 * x ][ 2 * y − 1] + pY[ 2 * x − 1 ][ 2 * y ] + 4 * pY[ 2 * x ][ 2 * y ] + pY[2*x+1][2*y]+pY[2*x][2*y+1]+4)>>3 (8-163)

- If the variable availL is TRUE, pDsY[ 0 ][ y ] for y = 1..nTbH − 1 is defined as follows:

pDsY[ 0 ][ y ] = ( pY[ 0 ][ 2 * y − 1 ] + pY[ −1 ][ 2 * y ] + 4 * pY[ 0 ][ 2 * y ] + pY[ 1 ][ 2 * y ] + pY[ 0 ][ 2 * y + 1 ] + 4 ) >> 3 (8-164)

- Otherwise, pDsY[ 0 ][ y ] for y = 1..nTbH − 1 is defined as follows:

pDsY[0][y] = (pY[0][2* y − 1] + 2* pY 0][2* y] + pY[0][ 2 * y + 1] + 2) >> 2 (8-165)

- If the variable availT is TRUE, pDsY[ x ][ 0 ] for x = 1..nTbW − 1 is defined as follows:

pDsY[ x ][ 0 ] = ( pY[ 2 * x ][ −1 ] + pY[ 2 * x − 1 ][ 0 ] + 4 * pY[ 2 * x ][ 0 ] + pY[ 2 * x + 1][0]+pY[2*x][1]+4)>>3 (8-166)

- Otherwise, pDsY[ x ][ 0 ] for x = 1..nTbW − 1 is defined as follows:

pDsY[ x ][ 0 ] = ( pY[ 2 * x − 1 ][ 0 ] + 2 * pY[ 2 * x ][0] + pY[ 2 * x + 1 ][0] + 2 ) >> 2 (8-167)

- If availL is TRUE and availT is TRUE, pDsY[ 0 ][ 0 ] is defined as follows:

pDsY[0][0]=(pY[0][−1]+pY[−1][0]+4*pY[0][0]+pY[1][0]+pY[0][ 1]+4)>>3 (8-168)

- Otherwise, if availL is TRUE and availT is FALSE, pDsY[ 0 ][ 0 ] is defined as follows:

pDsY[ 0 ][ 0 ] = ( pY[ −1 ][ 0 ] + 2 * pY[ 0 ][ 0 ] + pY[ 1 ][ 0 ] + 2 ) >> 2 (8-169)

- Otherwise, if availL is FALSE and availT is TRUE, pDsY[ 0 ][ 0 ] is defined as follows:

pDsY[ 0 ][ 0 ] = ( pY[ 0 ][ −1 ] + 2 * pY[ 0 ][ 0 ] + pY[ 0 ][ 1 ] + 2 ) >> 2 (8-170)

- Otherwise (availL is FALSE and availT is FALSE), pDsY[ 0 ][ 0 ] is defined as follows:

pDsY[ 0 ][ 0 ] = pY[ 0 ][ 0 ] (8-171)

- Otherwise, the following applies:

- pDsY[ x ][ y ] for x = 1..nTbW − 1, y = 0..nTbH − 1 is determined as follows:

pDsY[ x ][ y ] = ( pY[ 2 * x − 1 ][ 2 * y ] + pY[ 2 * x − 1 ][ 2 * y + 1 ] +
2* pY[ 2 * x ][ 2 * y ] + 2*pY[ 2 * x ][ 2 * y + 1 ] +
pY[ 2 * x + 1 ][ 2 * y ] +
pY[ 2 * x + 1 ][ 2 * y + 1 ] + 4 ) >> 3 (8-172)

- If the variable availL is TRUE, pDsY[ 0 ][ y ] for y = 0..nTbH − 1 is defined as follows:

pDsY[ 0 ][ y ] = ( pY[ −1 ][ 2 * y ] + pY[ −1 ][ 2 * y + 1 ] +
2* pY[ 0 ][ 2 * y ] + 2*pY[ 0 ][ 2*y + 1 ] +
pY[ 1 ][ 2 * y ] + pY[ 1 ][ 2 * y + 1 ] + 4 ) >> 3 (8-173)

- Otherwise, pDsY[ 0 ][ y ] for y = 0..nTbH − 1 is defined as follows:

= ( pY[ −3 ][ 0 ] + 2 * pY[ −2 ][ 0 ] + pY[ −1 ][ 0 ] + 2 ) >> 2 (8-177)

- Otherwise, the following applies:

= ( pY[ −1 ][ 2 * y ] + pY[ −1 ][ 2 * y + 1 ] +
2* pY[ −2 ][ 2 * y ] + 2*pY[ −2 ][ 2 * y + 1 ] + pY[ −3 ][ 2 * y ] + pY[ −3]
[ 2 * y + 1 ] + 4 ) >> 3 (8-178)

- If bCTUboundary is FALSE, the following applies:

= ( pY[2 * x ][−3] +pY[ 2 * x − 1 ][ −2 ] + 4 * pY[ 2 * x ][ −2 ] + pY[ 2 * x + 1 ][ −2 ] +pY[ 2 * x ][ −1 ] + 4 ) >> 3 (8-179)

- Otherwise (bCTUboundary is TRUE), the following applies:

= ( pY[ 2 * x − 1][ −1] + 2* pY[ 2 * x ][ −1 ] + pY[ 2 * x + 1 ][ −1 ] + 2 ) >> 2 (8-180)

- Otherwise:

- If availTL is TRUE and bCTUboundary is FALSE, the following applies:

= ( pY[ 0 ][ −3 ] +pY[ −1 ][ −2 ] + 4 * pY[ 0 ][ −2 ] + pY[ 1 ][ −2 ] + pY[ 0 ][ −1 ] + 4 ) >> 3 (8-181)

- Otherwise, if availTL is TRUE and bCTUboundary is TRUE, the following applies:

= ( pY[ −1 ][ −1 ] + 2* pY[ 0 ][ −1 ] + pY[ 1 ][−1] + 2 ) >> 2 (8-182)

- Otherwise, if availTL is FALSE and bCTUboundary is FALSE, the following applies:

= (pY[ 0 ][ −3 ] + 2 * pY[ 0 ][ −2 ] + pY[ 0 ][ −1 ] + 2 ) >> 2 (8-183)

- Otherwise (availTL is FALSE and bCTUboundary is TRUE), the following applies:

= pY[ 0 ][ −1 ] (8-184)

− Otherwise, the following applies:

- If bCTUboundary is FALSE, the following applies:

= (pY[2 * x − 1][−2] + pY[ 2 * x − 1][ −1 ] +2* pY[ 2 * x][−2 ] + 2*pY[ 2 * x ][ −1] + pY[ 2 * x + 1 ][ −2 ] + pY[ 2 * x + 1 ][ −1 ] + 4 ) >> 3 (8-185)

− Otherwise the parameter (bCTUboundary is TRUE), the following applies:

= ( pY[ 2 * x − 1 ][ −1 ] + 2* pY[ 2 * x ][ −1 ] + pY[ 2 * x + 1 ][ −1 ] + 2 ) >> 2 (8-186)

- Otherwise:

- If availTL is TRUE and bCTUboundary is FALSE, the following applies:

=( pY[− 1][−2] + pY[− 1][−1] + 2* pY[0][−2] + 2*pY[0][ −1] + pY[ 1 ][ − 2 ] + pY[ 1 ][ −1 ] + 4 ) >> 3 (8-187)

- Otherwise, if availTL is TRUE and bCTUboundary is TRUE, the following applies:

= ( pY[ −1][−1] + 2* pY[0][ −1] + pY[1][ −1 ] + 2 ) >> 2 (8-188)

- Otherwise, if availTL is FALSE and bCTUboundary is FALSE, the following applies:

= ( pY[ 0 ][ −2 ] + pY[ 0 ][ −1 ] + 1 ) >> 1 (8-189)

- Otherwise (availTL is FALSE and bCTUboundary is TRUE), the following applies:

= pY[ 0 ][ −1 ] (8-190)

7. Variables a, b and k are defined as follows:

- If numSampL is 0 and numSampT is 0, the following applies:

k = 0 (8-208) a = 0 (8-209) b = 1 << ( BitDepth _C − 1) (8-210)

- Otherwise, the following applies:

diff = maxY − minY (8-211)

- If diff is not 0, the following applies:

diffC = maxC − minC (8-212) x=Floor(Log2(diff)) (8-213) normDiff = ( ( diff << 4 ) >> x) & 15 (8-214) x+= (normDiff  !=  0) ? 10 (8-215) y = Floor( Log2( Abs ( diffC ) ) ) + 1 (8-216) a = ( diffC * ( divSigTable[ normDiff ] | 8 ) + 2 ^{y − 1} ) >> y (8-217) k = ( ( 3 + x − y ) < 1 ) ? 1 : 3 + x − y (8-218) a = ( ( 3 + x − y ) < 1 ) ? Sign(a)*15 : a (8-219) b = minC − ( ( a * minY ) >> k ) (8-220)

where the divSigTable[ ] parameter is specified as follows:

divSigTable[ ] = { 0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0 } (8-221)

- Otherwise (diff equals 0), the following applies:

k = 0 (8-222) a = 0 (8-223) b = minC (8-224)

8. Predicted samples predSamples[ x ][ y ] for x = 0..nTbW − 1, y = 0.. nTbH − 1 are determined as follows:

predSamples[ x ][ y ] = Clip1C( ( ( pDsY[ x ][ y ] * a ) >> k ) + b ) (8-225)

2.2.3.3 Various aspects of intra-frame prediction

The VTM4 document contains many intra-frame coding tools other than the HEVC coding standard, for example, the following features were included in the VVC coding standard test model 3 on top of the block tree structure.

- 67 intra-frame prediction modes with extension to wide-angle modes

- Block size and mode dependent interpolation filter with 4 taps

- Position dependent intra prediction combination (PDPC)

- Intra-frame prediction using cross-component linear model

- Intra-frame prediction using multiple reference lines

- Intra-frame division into subblocks

2.2.4 Intra-prediction mode in the VVC coding standard

2.2.4.1 Combined intra-interprediction (CIIP) mode

In the VTM4 document, when a CU is encoded in the combined mode, and if the CU contains at least 64 luminance samples (in other words, the product of the width of the CU and the height of the CU is at least 64), an additional flag is signaled to indicate whether combined inter/intra prediction (CIIP) is applied to the current CU.

To generate CIIP combined mode prediction, an intra-frame prediction mode is first determined based on two additional syntax elements. Up to four possible intra-prediction modes can be used: DC mode, planar mode, horizontal mode, or vertical mode. Then, inter- and intra-frame prediction signals are generated using regular intra- and inter-frame prediction procedures. Finally, weighted averaging of these inter- and intra-frame prediction signals is performed to obtain combined CIIP mode prediction.

2.2.4.2 Various aspects of intra-frame prediction

The VTM4 document contains many inter-prediction coding tools that differ from the HEVC coding standard, for example, the following features were included in the VVC coding standard test model 3 on top of the block tree structure.

- Affine interframe motion prediction

- Temporal prediction of motion vectors based on subblocks

- Adaptive motion vector resolution

- Motion compression based on 8x8 blocks for temporal motion prediction

- High precision motion vector storage (1/16 pel) and motion compensation using 8-tap luminance interpolation filter and 4-tap chrominance interpolation filter

- Triangular partition

- Combined mode with intra-frame and inter-frame prediction

- Merge mode with differences MVD (MMVD)

- Symmetric MVD difference coding

- Bidirectional optical flow

- Clarification of the motion vector on the side of the decoding device

- Weighted averaging with bidirectional prediction

2.2.5 In-circuit filters

There are only three in-loop filters in the VTM4 document. In addition to the deblocking filter and the sample adaptive offset filter (SAO), two types of filters used in the HEVC coding standard, the VTM4 document uses an adaptive loop filter (ALF). Filtering in a VTM4 document is done in the order of deblocking filter, SAO filter, and ALF filter.

In the VTM4 document, the filtering procedures in the SAO filter and deblocking filtering are almost the same as in the HEVC encoding standard.

In the VTM4 document, a new procedure was added called chroma scaling luma mode (previously known as adaptive in-loop reshaper). This new procedure is carried out before release.

2.2.6 Luminance with Chroma Scaling (LMCS, also called in-loop resampling)

In the VTM4 document, an encoding tool called luma mapping with chroma scaling (LMCS) is added as a new processing unit before edge filtering. The LMCS mode has two main components: 1) in-circuit display of the brightness component based on adaptive piecewise linear models; 2) applying the scaling of the remainder of the chrominance component, depending on the luminance component, to the chrominance components. Fig. 4 shows an implementation architecture of the LMCS mode from the point of view of a decoder. The light blue shaded blocks in Fig. 4 indicate that processing is applied to the displayed area; This includes inverse quantization, inverse transformation, intra-frame luminance prediction, and summation of luminance prediction with the remainder of the luminance component. The unshaded blocks in Fig. 4 indicate where processing is applied in the original (ie, unmapped) region; and these include edge filters such as deblocking filters, ALF filters and SAO filters, motion compensation prediction, intra-frame chrominance prediction, summing the chrominance prediction result with the chrominance residual, and storing the decoded pictures as reference pictures. Blocks shaded in light yellow in Fig. 4 are new functional blocks for the LMCS mode, including forward and inverse luminance signal mapping and a luminance-dependent chrominance scaling procedure. Like most other tools in the VVC coding standard, the LMCS scaling mode can be enabled/disabled at the sequence level using the SPS set flag.

2.2.6.1 Luminance display using a piecewise linear model

In-loop luminance mapping adjusts the dynamic range of the input signal by redistributing codewords across the dynamic range to improve compression efficiency. The luma mapping procedure uses the forward mapping function, FwdMap, and the corresponding inverse mapping function, InvMap. The FwdMap function is signaled using a piecewise linear model of 16 equal segments. The InvMap function does not need to be signaled, but is instead derived from the FwdMap function.

The luminance component display model is signaled at the tile group level. First, a presence flag is transmitted as signaling. If a luminance component display model is present in the current tile group, the parameters of the corresponding piecewise linear model are signaled. The piecewise linear model divides the dynamic range of the input signal into 16 equal segments, and for each segment, its linear mapping parameters are expressed using a number of codewords assigned to that segment. Let's take 10-bit input data as an example. Each of the 16 segments will have 64 codewords assigned to it by default. The signaled series of code words is used to calculate the scaling factor and adjust the display function for the corresponding segment according to this factor. At the tile group level, another LMCS scaling enable flag is signaled to indicate whether the LMCS mapping procedure shown in FIG. 4, to the current tile group.

Each i-th segment, i = 0 ... 15, of the piecewise linear model of the FwdMap function is defined by two input endpoints (breakpoints) InputPivot[] and two output (mapped) endpoints MappedPivot[].

The InputPivot[] and MappedPivot[] points are calculated as follows (assuming 10-bit video data):

1) OrgCW = 64

2) For i = 0:16, InputPivot[ i ] = i * OrgCW

3) For i=0:16, MappedPivot[i] is calculated as follows:

MappedPivot[0]=0;

for(i=0;i <16;i++)

MappedPivot[i + 1] = MappedPivot[i] + SignalledCW[i],

where the SignalledCW[i] values represent the number of codewords reported as signaling for the i-th segment.

As shown in FIG. 4, for a block encoded in the inter-prediction mode, motion-compensated prediction is performed on the mapped area. In other words, after calculating the predicted block with motion compensation based on reference signals in the DPB buffer, apply the FwdMap function to map the predicted luminance component block from the source region to the mapped region, . For a block encoded in intra-prediction mode, the FwdMap function is not used because intra-prediction is performed on the mapped region. After calculating the reconstructed block apply the InvMap function to transform the reconstructed luminance values from the mapped region back to the reconstructed luminance values in the original region, ( ). The InvMap function is applied to both luminance blocks - those encoded using intra- and inter-frame prediction.

The procedure for displaying the brightness component (direct and/or inverse display) can be carried out either using look-up tables (LUTs) or using real-time calculations (“on the fly”). If a LUT is used, then the tables And can be pre-computed and pre-stored for use at the tile group level, so that forward and backward mapping can be simply implemented as And , respectively. Alternatively, real-time (“on-the-fly”) calculations can be used. Let's take the direct mapping function FwdMap as an example. In order to determine the segment to which a certain sample of the brightness component belongs, the value of this sample is shifted to the right by 6 bits (which corresponds to 16 equal segments). The linear model parameters for that segment are then retrieved and calculations are applied on the fly to determine the displayed luminance value. Let i denote the segment index, a1, a2 denote the points InputPivot[i] and InputPivot[i+1], respectively, and b1, b2 denote the points MappedPivot[i] and MappedPivot[i+1], respectively. The FwdMap function is calculated as follows:

The InvMap function can be calculated on the fly in a similar way, except that it is necessary to apply conditional checks instead of simply shifting bits to the right when determining the segment to which the sample value in question belongs, since the segments in the mapped region are not of equal sizes.

2.2.6.2 Luminance-dependent scaling of the remaining chrominance component

Chroma residual scaling is designed to compensate for the interaction between the luma signal and its corresponding chrominance signals. Whether chrominance residual scaling is enabled or not is also reported as an alarm at the tile group level. If luminance display is enabled, and if dual-tree partitioning (also known as split chroma partitioning tree) is not applied to the current tile group, an additional flag is signaled to indicate whether luminance-dependent scaling of the remaining chroma partition is enabled or not. . When luma display is not used, or when bi-tree tiling is used in the current tile group, luma-dependent scaling of the remaining chrominance is not enabled. Further, luminance component-dependent scaling of the remaining chrominance component is always disabled for chrominance component blocks whose area is no larger than 4.

The scaling of the remaining chrominance component depends on the average value of the corresponding predicted block of the luminance component (for blocks of both types - with intra-frame and inter-frame prediction). Let's denote as the average value of the predicted block of the brightness component. Meaning calculated as follows:

1) Finding the index piecewise linear model, to which the value belongs based on the InvMap function.

2) = cScaleInv[ ], where cScaleInv[] denotes the precomputed LUT for 16 segments.

If the current block is encoded using the intra-frame prediction mode, the combined CIIP prediction mode, or the intra block copy (IBC) mode, otherwise known as current picture referencing (CPR) mode, the value calculated as the average of the luminance component values predicted in the intra-prediction mode, the combined CIIP mode, or the IBC copy mode; otherwise, the value is calculated as the average of the forward-mapped values of the luminance component in inter-frame prediction ( in Fig. 4). In contrast to the display of the brightness component, carried out on the basis of samples, the parameter has a constant value for the entire block of the chrominance component. Having , the scaling of the remaining chrominance component is applied as follows:

Encoder side:

Decoder side:

2.2.6.3 Corresponding working draft in JVET-M1001_v7 with application in JVET-N0220

The following description is based on a modified working draft from JVET-M1001 and the adoption in JVET-N0220. The modification in the accepted document JVET-N0220 is shown in bold with underlining.

Syntax tables

Semantics

B 7.4.3.1 Semantics of the RBSP sequence parameter set

Flag equal to 1 specifies that the luma display mode with chrominance scaling is used in the CVS sequence. The sps_lmcs_enabled_flag flag equal to 0 specifies that the chrominance scaling luma display mode is not used in the CVS sequence.

Flag equal to 1 specifies that the lmcs_data() parameter is present in the tile group header. A tile_group_lmcs_model_present_flag of 0 specifies that the lmcs_data() parameter is not present in the tile group header. When the tile_group_lmcs_model_present_flag is not present, it is set to 0.

Flag equal to 1 specifies that the luminance component display mode with chrominance component scaling is activated for the current group of tiles. The tile_group_lmcs_enabled_flag flag equal to 0 specifies that the luma component display mode with chroma scaling is not enabled for the current group of tiles. When the tile_group_lmcs_enabled_flag is not present, it is set to 0.

Flag equal to 1 specifies that chroma residual scaling is enabled for the current tile group. The tile_group_chroma_residual_scale_flag flag equal to 0 specifies that chroma residual scaling is not enabled for the current tile group. When the tile_group_chroma_residual_scale_flag is not present, it is set to 0.

In 7.4.5.4 Data semantics in the luminance component display mode with chrominance component scaling

Parameter specifies the minimum section (bit) index used in the procedure for constructing the display mode of the luminance component with scaling of the chrominance component. The value of the lmcs_min_bin_idx parameter must be in the range from 0 to 15 inclusive.

Parameter specifies the value of the difference between 15 and the maximum section (bit) index LmcsMaxBinIdx used in the procedure for constructing the luminance component display mode with chrominance component scaling. The value of the lmcs_delta_max_bin_idx parameter must be in the range from 0 to 15 inclusive. The value of the LmcsMaxBinIdx parameter is set equal to 15 − lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx must be no less than lmcs_min_bin_idx.

Parameter plus 1 specifies the number of bits used to represent the lmcs_delta_abs_cw[ i ] syntax. The value of the lmcs_delta_cw_prec_minus1 parameter must be in the range from 0 to BitDepthY − 2, inclusive.

Parameter specifies the absolute value of the increment (delta) of the codeword for the i-th section (bit).

Flag specifies the sign of the variable lmcsDeltaCW[ i ] as follows:

- If lmcs_delta_sign_cw_flag[ i ] is 0, the lmcsDeltaCW[ i ] parameter has a positive value.

- Otherwise ( lmcs_delta_sign_cw_flag[i] is not equal to 0), the lmcsDeltaCW[ i ] parameter has a negative value.

When the lmcs_delta_sign_cw_flag[ i ] flag is not present, it is set to 0.

The OrgCW variable is defined as follows:

OrgCW = (1 << BitDepth _Y )/16 (7-70)

The variable lmcsDeltaCW[i], with i = lmcs_min_bin_idx..LmcsMaxBinIdx, is defined as follows:

lmcsDeltaCW[ i ] = ( 1 − 2 * lmcs_delta_sign_cw_flag[ i ] ) * lmcs_delta_abs_cw[ i ] (7-71)

The variable lmcsCW[ i ] is defined as follows:

- For i = 0.. lmcs_min_bin_idx − 1, the variable lmcsCW[i] is set to 0.

- For i = lmcs_min_bin_idx..LmcsMaxBinIdx, the following applies:

lmcsCW[ i ] = OrgCW + lmcsDeltaCW[ i ] (7-72)

The value of the lmcsCW[i] variable must be in the range from (OrgCW>>3) to (OrgCW<<3 − 1), inclusive.

- For i = LmcsMaxBinIdx + 1..15, the variable lmcsCW[ i ] is set to 0.

The data bitstream compliance requirement is that the following condition be true:

The variable InputPivot[ i ], with i = 0..16, is defined as follows:

InputPivot[ i ] = i * OrgCW (7-74)

The variable LmcsPivot[ i ] for i = 0..16, the variables ScaleCoeff[ i ] and InvScaleCoeff[ i ] for i = 0..15 are defined as follows:

LmcsPivot[0]=0;

for(i=0;i<=15;i++) {

LmcsPivot[ i + 1 ] = LmcsPivot[ i ] + lmcsCW[ i ]

}

The variable ChromaScaleCoeff[ i ], with i = 0…15, is determined as follows:

}

The variables ClipRange, LmcsMinVal and LmcsMaxVal are defined as follows:

ClipRange = ((lmcs_min_bin_idx > 0) && ( LmcsMaxBinIdx < 15 ) (7-77) LmcsMinVal = 16 << (BitDepth _Y − 8)
LmcsMaxVal = 235 << (BitDepth _Y − 8) (7-78)
(7-79)

NOTE – The InputPivot[ i ] and LmcsPivot[ i ], ScaleCoeff[ i ], and InvScaleCoeff[ i ], ChromaScaleCoeff[ i ], ClipRange, LmcsMinVal and LmcsMaxVal arrays are updated only when the tile_group_lmcs_model_present_flag flag is 1. Thus, a model with scaling lmcs can be transmitted with an IRAP image, for example, but scaling lmcs is not activated for this IRAP image.

3. Disadvantages of existing implementation options

Today's LMCS/CCLM model configuration may have the following problems:

1. In the LMCS scaling encoding tool, the chrominance residual scaling factor is determined by the average of a colocated luminance block prediction, resulting in a delay in processing the chrominance samples when scaling the chroma residual in LMCS mode.

a) In case of single/shared tree, the delay is due to

(a) waiting for all predicted luminance samples of a complete luminance block to be available, and (b) averaging all predicted luminance samples resulting from (a).

b) In the case of a dual/split tree, the latency is even worse since the I-slices enable separate blocking structures for luma and chrominance components. Therefore, one chroma block can correspond to several luminance blocks, and one 4x4 chrominance block can correspond to a 64x64 luminance block. Thus, the worst case is that it may be necessary to wait for the remaining chrominance scaling factor for the current 4x4 chrominance block until all predicted samples in the full 64x64 luminance block are available. In short, the latency problem in the dual/split tree case can be much more severe.

2. In a CCLM encoding tool, the CCLM model's calculations for intra-chrominance prediction depend on the top and left reference samples for both the luma block and the chrominance block. And the CCLM predictions for a chroma block depend on co-located reconstructed luminance samples from the same CU. This may cause more latency when using dual/split trees.

- In the case of a double/split tree, one 4x4 chrominance block can correspond to a 64x64 luminance block. Thus, the worst case is that the CCLM procedure for the current chrominance block may have to wait until the full 64x64 luminance block has been reconstructed. This latency issue may be similar to the situation when using LMCS chroma scaling in a dual/split tree structure.

4. Examples of methods and options

To address this issue, we propose several ways to eliminate/reduce/limit the cross-component delay using luminance-dependent chrominance residual scaling, the CCLM model, and other encoding tools that rely on information from other chrominance components.

The detailed descriptions of the options below should be considered as examples to clarify general concepts. These options should not be interpreted in a narrow sense. Moreover, these options can be combined in any way.

Note that although the LMCS/CCLM model is explicitly mentioned in the sections below, these methods are also applicable to other encoding tools that rely on information from other color components. In addition to this, the terms "luma" and "chroma" mentioned below may be replaced by "first color component" and "second color component" respectively, such as "green (G ) component" and "blue/red (B/R) component" in RGB color format.

In the following discussion, the definition of “collocated sample/block” corresponds to the definition of collocated sample/block in the JVET-M1001 working draft of the VVC encoding standard. More specifically, in a 4:2:0 color format, suppose the top-left sample of a chrominance block is at position ( xTbC, yTbC ), then the position ( xTbY, yTbY ) of the top-left sample of a co-located luminance block is determined as follows thus: ( xTbY, yTbY ) = ( xTbC << 1, yTbC << 1 ). As illustrated in FIG. 5, the upper left sample of the current chrominance component block is located at the point (x=16,y=16) on the chrominance image, then the upper left sample of the luminance component block located at the same place is located at the point (x=32,y=32) of the luminance component image, regardless of the division into blocks of this located in the same place block of the luminance component in the luminance image. As another example, regarding the same color component, the position of the top left sample of a co-located block in the reference frame must match the position of the top left sample of the current block in the current frame, as illustrated in FIG. 6, suppose that the top left sample of the current block is located at point (x,y) in the current frame, then the top left sample of the block located at the same location relative to the current block has the same position (x,y) in the reference frame.

In the following discussion, the "corresponding block" may occupy a position different from the current block. As an example, there may be a motion shift between the current block and its corresponding block in the reference frame. As illustrated in FIG. 6, suppose that the current block is located at point (x,y) in the current frame and has a motion vector (mv _x , mv _y ), then the corresponding block relative to this current block can be located at point (x+mv _x ,y+mv _y ) in the reference frame. And for a block encoded in the IBC copy mode, a luma block located at the same location (indicated by a zero vector) and its corresponding luma block (indicated by a non-zero BV vector) may be located at different locations in the reference frame. As another example, when the partition of a luma block is not the same as the partition of a chroma block (in a dual-tree split of I-slices), a colocated luma block relative to the current chroma block may belong to a larger luma block, such that depends on the size of the partition units of the superimposed luminance coding block covering the upper left sample of the co-located luminance block. As illustrated in FIG. 5, the bold rectangle is assumed to represent block splits such that the 64x64 luma block is first split into a binary tree (BT), and then the right side of the 64x64 luma block is further split into a ternary tree (TT), resulting in three blocks of the brightness component with dimensions equal to 32x16, 32x32, 32x16, respectively. Thus, if you look at the top left sample (x=32, y=32) of the same location luma block relative to the current chrominance block, this sample belongs to the central 32x32 luma block from the ternary (TT) tree partition . In such a case, we call the corresponding luminance component block covering the upper left sample of the luminance component block located at the same location as the “corresponding luminance component block”. Therefore, in this example, the top left sample of the corresponding luminance component block is located at the point with coordinates (x=32, y=16).

Hereinafter, the concept of “decoder-side motion vector derivation” (DMVD) is used to represent the bidirectional optical flow BDOF (also called BIO) or/and the decoder-side motion vector refinement method ( DMVR (decode-side motion vector refinement)) and/or frame rate up-conversion method (FRUC) and/or another method that allows one to refine the motion vector and/or the predicted sample in the decoding device.

Eliminate delay in LMCS chroma scaling and CCLM model calculations

1. It is proposed that for a block encoded in the intra-frame prediction mode, one or more reference samples from the current block in the reference frames can be used to determine the chrominance residual scaling factor in the LMCS scaling mode.

a) In one example, the luma reference samples can be directly used to determine the scaling factor of the chrominance residual.

i. Alternatively, interpolation may be first applied to the reference samples, and the interpolated samples may be used to determine a scaling factor for the chroma residual.

ii. Alternatively, reference samples in different reference frames can be used to determine the final reference samples used to determine the chroma residual scaling factor.

1) In one example, the above methods may be applied to bidirectionally encoded blocks.

iii. In one example, the reference sample intensities may be converted to the reshaped region before being used to determine the chrominance residual scaling factor.

iv. In one example, a linear combination of reference samples can be used to determine the scaling factor of the chroma residual.

1) For example, the expression a×S+b can be used to determine the scaling factor of the chrominance component residual, where S denotes the reference sample, a and b parameters. In one example, the parameters a and b may be determined by Localized Illuminate Compensation (LIC).

b) In one example, the location of the luminance component reference samples in the reference frame may depend on the motion vector(s) for the current block.

i. In one example, the reference samples belong to a luminance reference block that is in the reference image and has the same width and height as the current luminance block. The position of a reference luminance sample in a reference image can be calculated as the position of a corresponding luminance sample in the current image plus a motion vector.

ii. In one example, the positions of the reference luminance samples may be determined based on the position of the top left (or center or bottom right) sample of the current luminance block and a motion vector for that current block, where said sample is referred to as the corresponding luminance sample in the reference frame .

1) In one example, an integer motion vector can be used to determine the corresponding luminance component sample in the reference frame. In one example, to obtain an integer motion vector, the motion vector associated with one block may be either rounded toward zero or rounded away from zero.

2) Alternatively, a fractional motion vector may be used to determine the corresponding luminance component sample in the reference frame, so an interpolation procedure may be required to determine the fractional reference samples.

iii. Alternatively, the positions of the reference luminance samples may be determined based on the position of the top left (or center or bottom right) sample of the current luminance block.

iv. Alternatively, a plurality of corresponding luminance samples at certain predetermined positions in the reference frame may be selected to calculate a scaling factor for the chrominance residual.

c) In one example, the median or average value of a plurality of luminance reference samples may be used to determine a scaling factor for the chroma residual.

d) In one example, luminance reference samples in predetermined reference frames can be used to determine a scaling factor for the chrominance residual.

i. In one example, the predetermined reference frame may be a frame with a reference index of zero in the reference picture list 0.

ii. Alternatively, the reference index and/or list of reference pictures for a predetermined reference frame may be signaled at the sequence/picture/tile group/slice/tile/line of CTUs/video unit level.

iii. Alternatively, reference luminance samples in multiple reference frames may be determined and the average or weighted averages may be used to obtain a scaling factor for the remaining chrominance component.

2. It is proposed that whether and how to determine the chrominance residual scaling factor from the luminance samples in the LMCS scaling mode may depend on whether bidirectional prediction is applied for the current block.

a) In one example, the chrominance residual scaling factor is determined for each prediction direction individually.

3. It is proposed that whether and how to determine the chrominance residual scaling factor from the luminance samples in the LMCS scaling mode may depend on whether subblock-based prediction is applied.

a) In one example, subblock-based prediction is affine prediction;

b) In one example, the subblock-based prediction method is an Alternative Temporal Motion Vector Prediction (ATMVP) method.

c) In one example, the chrominance residual scaling factor is determined for each subblock individually.

d) In one example, the chrominance residual scaling factor is determined for the entire block, even if it is predicted across subblocks.

i. In one example, the motion vector for one selected subblock (eg, the top left subblock) can be used to identify reference samples of the current block, as described in Section 1.

4. It is proposed that the predicted luminance values used to determine the chroma residual scaling factor may be intermediate predicted luminance values instead of the final predicted luminance values.

a) In one example, the predicted luminance values before the Bi-Directional Optical Flow (BDOF, also called BIO) procedure can be used to determine the scaling factor of the remaining chrominance component.

b) In one example, the predicted luminance values prior to the Decoder-side Motion Vector Refinement (DMVR) procedure can be used to determine the scaling factor of the remaining chrominance component.

c) In one example, the predicted luminance values before the LIC compensation procedure can be used to determine the scaling factor for the remaining chrominance component.

d) In one example, the predicted luminance values prior to the Prediction Refinement Optical Flow (PROF) procedure, as proposed in JVET-N0236, can be used to determine the scaling factor of the chrominance residual.

5. Intermediate motion vectors can be used to identify reference samples.

a) In one example, the motion vector prior to the BDOF flow procedure and/or DMVR refinement and/or other DMVD methods may be used to identify reference samples.

b) In one example, the motion vector prior to the predicted optical flow (PROF) refinement procedure, as proposed in JVET-N0236, can be used to identify reference samples.

6. The above methods may be applicable when the current block is encoded in inter-frame prediction mode.

7. It is proposed that for a block encoded in the IBC copy mode, one or more reference samples in the reference block of the current frame can be used to determine the chroma residual scaling factor in the LMCS scaling mode. When a block is encoded in IBC mode, the term "motion vector" may be equivalent to the term "block vector", where this reference picture is set as the current picture.

a) In one example, the reference sample belongs to a reference block that is in the current image and has the same width and height as the current block. The position of a reference sample can be calculated as the position of its corresponding sample plus a motion vector.

b) In one example, the positions of the luminance reference samples may be determined by the position of the top left (or center or bottom right) sample of the current luminance block plus a motion vector.

c) Alternatively, the positions of the luminance reference samples may be determined by the position of the top left (or center or bottom right) sample of the current luminance block plus the block vector of the current block.

d) Alternatively, several corresponding luminance samples at some predetermined positions in the reference region of the current luminance block may be selected to calculate a chrominance residual scaling factor.

e) In one example, several corresponding luminance samples may be calculated using a function to determine a scaling factor for the chrominance residual.

i. For example, the median or average value of several corresponding luminance samples may be calculated to determine a scaling factor for the chrominance residual.

f) In one example, the reference sample intensities may be converted to the reshaped region before being used to determine the chroma residual scaling factor.

i. Alternatively, the reference sample intensities can be converted to the original domain before being used to determine the chroma residual scaling factor

8. It is proposed that one or more predicted/reconstructed samples located at the identified position(s) of the current luminance block in the current frame can be used to determine the chroma residual scaling factor for the current chrominance block in the LMCS scaling mode.

a) In one example, if the current block is inter-predictive encoded, a predicted (or reconstructed) luminance sample located at the center of the current luminance block may be selected to determine a chrominance residual scaling factor.

b) In one example, the average of the first MxN predicted (or reconstructed) luminance samples may be selected to determine the scaling factor of the remaining chrominance component, where MxN may be less than the size (width x height product) of a co-located luminance block .

9. It is proposed that the CCLM model calculation procedure, in whole or in part, can be used to determine the chrominance residual scaling factor for a chrominance block in the LMCS scaling mode.

a) In one example, reference samples located at identified positions of adjacent luminance samples of a colocated luminance block during the CCLM model parameter determination procedure may be used to determine the scaling factor of the chrominance residual.

i. In one example, these reference samples can be used directly.

ii. Alternatively, subsampling may be applied to these reference samples, and the subsampled reference samples may then be used.

b) In one example, the K samples of the S reference samples selected for the CCLM model calculations can be used to determine the chroma residual scaling factor in the LMCS scaling mode. For example, K is 1 and S is 4.

c) In one example, the average/minimum/maximum value of the reference samples from a colocated luminance block in the CCLM model-based mode can be used to determine the chrominance residual scaling factor in the LMCS scaling mode.

10. How exactly to select samples to determine the scaling factor of the remaining chrominance component may depend on the encoded information of the current block.

a) This encoded information may include a QP parameter, encoding mode indication, POC number, intra prediction mode indication, motion information, etc.

b) In one example, for blocks coded in IBC copy mode or non-IBC copy mode (Non-IBC), the sample selection method may be different.

c) In one example, the sample selection method may be different based on reference picture information, such as the distance in POC numbers between the reference pictures and the current picture.

11. It is proposed that the chrominance residual scaling factor and/or CCLM model calculation may depend on neighboring samples of a corresponding luminance block covering the top left sample of a co-located luminance block.

a) A “corresponding luminance coding block” may be defined as a coding block covering the upper left position of a co-located luminance coding block.

i. In FIG. 5 shows an example in which, for an intra-predictive encoded chrominance block in the case of a dual tree, the CTU partition for the chrominance component may be different from the CTU partition for the luma component. First, a “corresponding luminance coding block” covering the upper left sample of a co-located luminance block with respect to the current chrominance block is isolated. Then, using the size information of this "corresponding luminance coding block", the upper left sample of the considered "corresponding luminance coding block", this upper left luminance sample of the "corresponding luminance coding block" covering the upper left sample located at in the same place of the brightness component block, located at the point with coordinates (x=32, y=16).

b) In one example, reconstructed samples not in the specified "corresponding luma coding block" may be used to determine the chrominance residual scaling factor and/or calculate the CCLM model.

i. In one example, the reconstructed samples adjacent to the "corresponding luminance coding block" may be used to determine the chrominance residual scaling factor and/or calculate the CCLM model.

1) In one example, N samples located in the left-adjacent columns and/or in the top-adjacent rows of the “corresponding luma coding block” may be used to determine the chrominance residual scaling factor and/or calculate the CCLM model, where N= 1...2W+2H, W and H denote the width and height of the “corresponding luminance component encoding block”.

a) Assume that the top left sample of the "corresponding luminance coding block" is at the point (xCb, yCb), then in one example, the upper adjacent luminance sample may be at the point (xCb + W/2, yCb - 1) , or (xCb -1, yCb - 1). In an alternative example, the luminance component sample next to the left may be at the point (xCb+W-1, yCb-1).

b) In one example, the position(s) of the adjacent sample(s) may be fixed, and/or be in a specified test order.

2) In one example, 1 of the N adjacent samples may be selected to determine the chroma residual scaling factor and/or calculate the CCLM model. Suppose that N=3, and the order of checking three adjacent readings is (xCb -1, yCb - H -1), (xCb + W/2, yCb - 1), (xCb -1, yCb - 1), then the first available adjacent sample in the check list can be selected to determine the scaling factor of the chroma residual.

3) In one example, the median or average value of the N samples located in the left-adjacent columns and/or top-adjacent rows of the “corresponding luma coding block” may be used to determine the chrominance residual scaling factor and/or calculate the CCLM model, where N=1...2W+2H, W and H denote the width and height of the “corresponding luminance component encoding block”.

c) In one example, whether to scale the chrominance residual may depend on the "available" neighboring samples of the corresponding luminance block.

i. In one example, the "availability" of neighboring samples may depend on the encoding mode of the current block/sub-block and/or the encoding mode of the adjacent sample.

1) In one example, for a block encoded in the inter-prediction mode, adjacent samples encoded in the intra-prediction mode and/or in the IBC copy mode and/or in the combined CIIP prediction mode and/or in the LIC compensation mode may be considered " inaccessible."

2) In one example, for a block encoded in the inter-frame prediction mode, neighboring samples using a diffusion filter and/or a two-way filter and/or a Hadamard transform filter may be considered "unavailable."

ii. In one example, the "availability" of neighboring samples may depend on the width and/or height of the current image/tile/group of tiles/VPDU/slice.

1) In one example, if a neighboring block is located outside the current image, then it is considered "inaccessible".

iii. In one example, when there are no "available" neighboring samples, scaling of the chroma residual may not be valid.

iv. In one example, when the number of "available" neighboring samples is less than K (K >= 1), scaling the chroma residual may not be valid.

v. Alternatively, the unavailable adjacent sample can be replaced with a fixed default value, either by padding or substitution, so that scaling of the remaining chrominance component can always be applied.

i) In one example, if an adjacent sample is not available, then it can be replaced by 1 << (bitDepth − 1), where the bitDepth parameter specifies the bit depth of the luma/chrominance samples.

2) Alternatively, if a neighboring sample is not available, then it can be replaced by padding from surrounding samples that are left/right/top/bottom neighbors.

3) Alternatively, if a neighbor sample is not available, then it can be replaced by substituting the first available neighbor sample in a predefined test order.

d) In one example, the filtered/mapped samples adjacent to the "corresponding luma coding block" may be used to determine the chrominance residual scaling factor and/or calculate the CCLM model.

i. In one example, the filtering/mapping procedure may comprise support smoothing filtering for intra-prediction blocks, post-filtering such as applying a two-way filter, a Hadamard transform filter, direct mapping of the reshaped region, and the like.

Restrictions on whether chroma residual scaling and/or CCLM is applied or not

12. It is proposed that whether chrominance residual scaling or CCLM is applied or not may depend on the partitioning of the corresponding and/or co-located luminance block.

a) In one example, whether or not to activate tools using cross-component information may depend on the number of CUs/PUs/TUs within a colocated luminance (eg, Y or G) component block .

i. In one example, if the number of CU units/PU units/TU units within a colocated luma (eg, Y or G) component block exceeds a threshold number, such tools may not be activated.

ii. Alternatively, whether or not to activate tools using cross-component information may depend on the depth of the partitioning tree.

1) In one example, if the maximum (or minimum or average or whatever) quadtree depth for CUs within a colocated luminance block exceeds a threshold, such tools may not be activated (deactivated) .

2) In one example, if the maximum (or minimum or average or whatever) binary (BT) and/or ternary (TT) tree depth for CUs within a co-located luminance block exceeds a threshold, such tools may not be activated (activation cancelled).

iii. Alternatively, in addition, whether or not to activate tools using cross-component information may depend on the chrominance block sizes.

iv. Alternatively, whether or not to activate tools using cross-component information may depend on whether a colocated luminance component block spans multiple VPDU/predefined area unit sizes.

v. The thresholds in the above discussion may be fixed numbers or may be communicated as alarms or may depend on standard profiles/levels/tiers.

b) In one example, if a luminance block located at the same location relative to the current chrominance block is split into multiple division levels (eg, as shown in FIG. 7), then scaling the remaining chroma and/or applying a CCLM model may be forbidden.

i. Alternatively, if a co-located luma block relative to a chrominance block is not partitioned (eg, within a single CU/TU/PU), then chrominance residual scaling and/or a CCLM model can be applied.

c) In one example, if a luminance block located at the same location relative to the current chrominance block contains more than M CUs/PUs/TUs, then scaling of the remaining chrominance and/or application of the CCLM model may be prohibited.

i. In one example, M can be an integer greater than 1.

ii. In one example, M may depend on whether it is a CCLM model or a chroma residual scaling procedure.

iii. The M number may be a fixed number or may be signaled or may depend on standard profiles/layers/tiers

d) The above CUs within a co-located luminance component block can be interpreted as all CUs within a co-located luminance component block. Alternatively, CUs within a colocated luminance block may be interpreted as just a portion of the CUs within that colocated luminance block, such as CUs along the boundary of that colocated luminance block. component.

e) The above-mentioned CUs within a co-located luminance component block can be interpreted as CU subunits or subblocks.

i. For example, CU subunits or subunits may be used in ATMVP prediction mode;

ii. For example, CU subunits or subblocks can be used in affine prediction;

iii. For example, CU subunits or subblocks can be used in Intra Sub-Partition (ISP) mode.

f) In one example, if the CU unit/PU unit/TU unit covering the top left luminance sample of a luminance block located at the same location is larger than the specified luminance block size, then scaling the chrominance residual and/or applying a CCLM model may be prohibited.

i. In the example shown in FIG. 8, a luminance block located at the same location is 32x32, but is within a corresponding 64x64 luminance block, then if the specified luminance block size is 32x64, then scaling of the remaining chrominance and/or application of the CCLM model is prohibited

ii. Alternatively, if a colocated luma block relative to the current chrominance block is not divided, and the corresponding luma block covering the top left luminance sample of the colocated luma block is completely contained within the bounding box, then For the current chroma block, chroma residual scaling and/or a CCLM model can be applied. The bounding box may be in the shape of a rectangle of width W and height H, denoted WxH, as shown in FIG. 9, the corresponding luminance component block has a width of 32 and a height of 64, and the bounding box has a width of 40 and a height of 70.

1) In one example, the size WxH of the bounding box may be determined according to the width and/or height of the CTU, or according to the width and/or height of the CU, or according to arbitrary values.

g) In one example, if a co-located luminance block relative to the current chrominance block is partitioned into multiple division levels, then only the predicted samples (or reconstructed samples) within a predefined partition unit of the co-located luminance block are used to determine the scaling factor for the remaining chrominance component in the LMCS scaling mode.

i. In one example, the average of all predicted samples (or reconstructed samples) in the first partition unit of a colocated luma block is used to determine the scaling factor of the remaining chrominance component in the LMCS scaling mode.

ii. Alternatively, the top left predicted sample (or reconstructed sample) in the first partition unit of the co-located luma block is used to determine the scaling factor of the remaining chrominance component in the LMCS scaling mode.

iii. Alternatively, the central predicted sample (or reconstructed sample) in the first partition unit of a co-located luminance block is used to determine the scaling factor of the chroma remainder in the LMCS scaling mode.

h) It is proposed that whether and how to apply cross-component tools such as the CCLM model and LMCS scaling may depend on the encoding mode(s) of one or more luminance component CUs covering at least one sample in a co-located block of the brightness component.

i. For example, cross-component tools are not activated if one or more luma CUs that cover at least one sample of a colocated luma block are affine-mode encoded;

ii. For example, cross-component tools are not activated if one or more luminance CUs that cover at least one sample of a colocated luminance block are encoded in a bidirectional predictive mode;

iii. For example, cross-component tools are not activated if one or more luma CUs that cover at least one sample of a colocated luma block are encoded in the BDOF stream mode;

iv. For example, cross-component tools are not activated if one or more luminance CUs that cover at least one sample of a colocated luminance block are encoded using DMVR refinement;

v. For example, cross-component tools are not activated if one or more luminance CUs that cover at least one sample of a colocated luminance block are encoded using an affine matrix prediction mode, as proposed in JVET-N0217 ;

vi. For example, cross-component tools are not activated if one or more luminance CUs that cover at least one sample of a colocated luminance block are encoded using an inter-prediction mode;

vii. For example, cross-component tools are not activated if one or more luma CUs that cover at least one sample of a colocated luma block are encoded using the ISP split mode;

viii. In one example, the words “one or more luminance component CUs that cover at least one sample of a colocated luminance component block” may refer to a corresponding luminance component block.

i) When CCLM/LMCS mode is disabled, CCLM/LMCS mode use signaling may be skipped.

j) As used herein, CCLM may refer to any of the CCLM modes, including LM mode, LM-T mode, and LM-L mode.

13. It is proposed that the decision of whether and how to apply cross-component tools such as the CCLM model and LMCS scaling can be made for a portion of the chrominance block.

a) In one example, whether and how to apply cross-component tools such as the CCLM model and LMCS scaling is decided at the chrominance subblock level.

i. In one example, a chrominance subblock is defined as a 2x2 or 4x4 block in a chrominance CU.

ii. In one example, for a chrominance subblock, when the corresponding luminance coding block relative to the current chroma CU covers all samples of the corresponding block or subblock, the CCLM model may be applied.

iii. In one example, for a chrominance subblock, when not all samples of the corresponding block are covered by the corresponding luminance coding block relative to the current chroma CU, the CCLM model is not applied.

iv. In one example, the CCLM model or LMCS scaling parameters are determined for each chroma sub-block, considering that sub-block as a chrominance CU.

v. In one example, when a CCLM model or LMCS scaling is applied to a chrominance subblock, samples from a colocated block may be used.

Applicability of chrominance residual scaling in LMCS mode

14. It is proposed that whether luma-dependent scaling of the chrominance residual can be applied can be signaled at another syntactic level in addition to the tile group header, as specified in JVET-M1001.

a) For example, a flag can be signaled at the sequence level (e.g. in an SPS set), at the picture level (e.g. in a PPS set or in a picture header), at the slice level (e.g. in a slice header), at the tile level, at the line of units level CTU, at unit level CTU, at unit level CU. Flag equal to 1 specifies that chrominance residual scaling is enabled for CUs below the signaled syntax level. Flag equal to 0 specifies that chrominance residual scaling is not enabled for a level below the signaled syntax level. When the chroma_residual_scale_flag is not present, it is set to 0.

b) In one example, if the chrominance residual scaling is limited at the split node level, then the flag may not be signaled and is set to 0 for CUs covered by a split node. In one example, the split node may be a CTU (the CTU is considered the root node in a quaternary tree split).

c) In one example, if chrominance residual scaling is limited for a chrominance block size of no more than 32x32, then the flag may not be transmitted in the form of signaling and then it is recognized as equal to 0 for the block size of the chrominance component not exceeding 32x32.

Applicability of CCLM based regime

15. It is proposed that whether a CCLM-based mode can be applied may be signaled at other syntax levels in addition to the sps set level as specified in JVET-M1001.

a) For example, this may be signaled at the picture level (eg in the PPS set or picture header), at the slice level (eg in the slice header), at the tile group level (eg in the tile group header), at tile level, at CTU line level, at CTU level, at CU level.

b) In one example, the flag may not be signaled and is then set to 0 if the CCLM model cannot be applied.

i. In one example, if chroma remainder scaling is limited to a chrominance block size of no more than 8x8, then the flag may not be transmitted in the form of signaling and then it is recognized as equal to 0 for chrominance component block sizes no larger than 8x8.

Unification of determination of the scaling factor of the remainder of the chrominance component for the intra-frame prediction mode and the inter-frame prediction mode

16. The chrominance residual scaling factor can be determined after encoding/decoding a luminance block and can be stored and used for subsequent encoded blocks.

a) In one example, some predicted samples and/and intermediate predicted samples and/and reconstructed samples and/and reconstructed samples before loop filtering (for example, before processing by a deblocking filter and/or an SAO filter and/or a two-way filter and/or a filter based on the Hadamard transform and/or an ALF filter) in the luma block can be used to determine the scaling factor of the remaining chrominance component.

i. For example, partial samples in the bottom row and/or in the right column of the luminance block may be used to determine the scaling factor for the chrominance component residual.

b) In the case of a single tree, when encoding a block in intra-prediction mode and/or IBC copy mode and/or inter-prediction mode, the found chrominance residual scaling factor for neighboring blocks can be used to determine the scaling factor for the current block.

i. In one example, certain neighboring blocks may be examined in order, and the first available chrominance residual scale factor may be used for the current block.

ii. In one example, some neighboring blocks may be examined in order, and a scale factor may be determined based on the first K available neighboring chrominance component residual scale factors.

iii. In one example, for a block encoded in the inter-prediction mode and/or the combined CIIP prediction mode, if an adjacent block is encoded in the intra-prediction mode and/or the IBC copy mode and/or the combined CIIP prediction mode, the residual chrominance scaling factor component for this neighboring block may be considered "unreachable".

iv. In one example, adjacent blocks can be checked in order from left (or top left) -> to top (or top right).

1) Alternatively, adjacent blocks can be checked in order from top (or top right) -> to left (or top left.

c) In the case of a split tree, when encoding a chrominance block, the corresponding luminance block may first be identified. The found scaling factor for the remaining chrominance component for its neighboring blocks (eg, the corresponding luma component block) can then be used to determine the scaling factor for the current block.

i. In one example, certain neighboring blocks may be examined in order, and the first available chrominance residual scale factor may be used for the current block.

ii. In one example, certain neighboring blocks may be examined in order, and the scaling factor may be determined based on the first K available neighboring chrominance component residual scaling factors.

d) Adjacent blocks can be checked in a predefined order.

i. In one example, adjacent blocks can be checked in order from left (or top left) -> to top (or top right).

ii. In one example, adjacent blocks can be checked in order from top (or top right) -> to left (or top left).

iii. In one example, adjacent blocks can be checked in the order bottom left -> left -> top right -> top -> top left.

iv. In one example, adjacent blocks can be checked in the order left -> top -> top right -> bottom left -> top left.

e) In one example, whether to apply chroma residual scaling may depend on the "availability" of an adjacent block.

i. In one example, when there is no "available" neighboring block, scaling the remaining chrominance component may not be valid.

ii. In one example, when the number of "available" neighboring blocks is less than K (K >= 1), scaling the remaining chrominance component may not be valid.

iii. Alternatively, when there is no "available" adjacent block, a default value can be taken as the scaling factor for the remaining chrominance component.

1) In one example, the default value of 1<< (BitDepth -1) can be used to determine the scaling factor for the remaining chrominance component.

5. Examples of implementation of the proposed technology

In FIG. 10 is a block diagram of video processing hardware 1000. This apparatus 1000 may be used to implement one or more of the methods described herein. The hardware 1000 may be a smartphone, a tablet computer, a conventional computer, an Internet of Things (IoT) receiver, etc. The hardware 1000 may include one or more processors 1002, one or more storage devices 1004, and video processing hardware 1006. Processor(s) 1002 may be configured to implement one or more of the methods (including, but not limited to, methods 800 and 900) described herein. Storage device(s) 1004 may be used to store data and code used to implement the methods and technology described herein. Video processing equipment 1006 may be used to implement, in hardware, some of the methods and technologies described herein.

In some embodiments, video encoding methods may be implemented using an apparatus implemented on a hardware platform, as described in connection with FIG. 10.

In FIG. 11 shows the logic diagram of an example method 1100 for defining a linear model for cross-component forecasting in accordance with the proposed technology. This method 1100 comprises, at step 1110, converting between the current video block and a data bitstream representation of that current video block, wherein the conversion process determines a second set of color component values for the current video block from the first set of color component values included in one or multiple reference frames, where the first set of color component values can be used in a linear model during the video encoding stage.

Some options may be described using the following clause-by-clause format.

1. A video processing method containing:

performing a conversion between the current video block and a representation of this current video block as a stream of data bits, where, in the conversion process, a second set of color component values for the current video block is determined from a first set of color component values included in one or more reference frames, where the first set of values color components can be used in a linear model at the video encoding stage.

2. The method according to article 1, characterized in that the first set of color component values is interpolated before being used in the linear model at the video encoding stage.

3. A method according to any one or more of Articles 1-2, characterized in that a linear combination of the first set of color component values can be used as parameters of the linear model.

4. The method according to article 1, characterized in that the positions of the first set of color component values included in one or more reference frames are selected based at least in part on motion information of the current video block.

5. The method according to Article 4, characterized in that the position of the luminance component value in one or more reference frames is calculated based on the position of the corresponding luminance component value in the current video block and motion information of the current video block.

6. The method according to Article 5, characterized in that the position of said corresponding luminance component value is the position of the upper left sample, the central sample or the lower right sample in the current video block.

7. The method according to Article 6, characterized in that the motion information of the current video block corresponds to an integer motion vector or a fractional motion vector.

8. The method according to Article 7, characterized in that the fractional motion vector is determined using the fractional value of the luminance component in one or more reference frames.

9. The method according to Article 7, characterized in that the integer motion vector is determined by rounding towards zero or away from zero.

10. The method according to Article 1, characterized in that the positions of the first set of color component values included in one or more reference frames are predetermined positions.

11. The method according to any one or more of clauses 1-10, characterized in that the median or average value of the first set of color component values is used to determine the second set of color component values for the current video block.

12. A method according to any one or more of Articles 1 to 11, characterized in that said one or more reference frames are predefined reference frames.

13. The method according to Article 12, characterized in that the set of predetermined reference pictures contains a frame with a reference index from the list of reference pictures.

14. The method according to Article 13, characterized in that the reference index is zero and the list of reference images is zero.

15. The method according to article 13, characterized in that the reference index and/or the list of reference pictures is signaled in a data bitstream representation associated with one or more objects from the following list: sequence, image, tile, group, slice , a string of coding tree units, or a video block.

16. The method according to article 1, characterized in that the second set of color component values for the current video block is determined on the basis of a mathematical average or weighted average for the first set of color component values included in one or more reference frames.

17. The method according to Clause 1, wherein the second set of color component values for the current video block is obtained selectively based on the first set of color component values included in one or more reference frames, based on whether the current video block is encoded in a bidirectional predictive mode.

18. The method according to article 17, characterized in that the second set of color component values for the current video block is individually determined for each prediction direction for the first set of color component values.

19. The method according to article 1, characterized in that the second set of color component values for the current video block is selectively determined based on the first set of color component values included in one or more reference frames, depending on whether the current video block is associated with prediction based on subblocks.

20. The method according to Article 1, characterized in that the subblock-based prediction corresponds to affine prediction or alternative time motion vector prediction (ATMVP).

21. The method according to any one or more of articles 19-20, characterized in that the second set of color component values for the current video block is determined for individual subblocks.

22. The method according to any one or more of clauses 19-21, characterized in that the second set of color component values for the current video block is determined for the entire current video block independent of the subblock-based prediction.

23. The method according to any one or more of clauses 19-22, characterized in that the first set of color component values included in the one or more reference frames is selected based at least in part on a motion vector for a subblock of the current video block.

24. A method according to any one or more of Articles 1-23, characterized in that the first set of color component values included in one or more reference frames represents intermediate color component values.

25. A method according to any one or more of Articles 1-24, characterized in that said video encoding step precedes another video encoding step.

26. The method according to article 25, characterized in that the first set of color component values included in one or more reference frames is selected based at least in part on an intermediate motion vector of the current video block or subblock of that current video block, and characterized in that this intermediate motion vector is calculated before said other video encoding step.

27. The method according to any one or more of Articles 24-26, wherein said other video encoding step comprises or a combination of the following steps: a bidirectional optical flow (BDOF) step, a decoder-side motion vector refinement (DMVR) step ,Predicted Optical Flow (PROF) refinement stage.

28. A method according to any one or more of clauses 1-27, characterized in that the first set of chroma values included in the one or more reference frames corresponds to MxN chroma values associated with the corresponding luminance component block.

29. The method according to Article 28, characterized in that the corresponding luminance component block is a luminance component block located at the same location relative to the current video block.

30. The method according to Article 29, characterized in that the product of M by N is less than the product of the block width by the block height for the luminance component block located in the same place relative to the current video block.

31. A method according to any one or more of Articles 27-30, characterized in that the first set of color component values included in one or more reference frames corresponds to at least a portion of the reference samples identified in the positions of adjacent luminance component samples of the located in the same place of the luminance component block.

32. A method according to any one or more of clauses 1-31, characterized in that the first set of color component values is downsampled before being used in the linear model of the video encoding step.

33. The method of Clause 1, wherein the second set of color component values of the current video block is selected based at least in part on one or more of the following information regarding the current video block: a quantization parameter, an encoding mode, or a picture order count (POC)).

34. The method according to Article 31, characterized in that the positions of adjacent luminance component samples are such that the upper left sample of a luminance component block located at the same location is covered.

35. The method according to article 28, characterized in that the first set of color component values included in one or more reference frames corresponds to at least a portion of the reference samples identified at positions outside the corresponding luma component block.

36. The method according to clause 28, characterized in that the second set of chroma values for the current video block is selectively obtained from the first set of chroma values included in one or more reference frames based on the availability of adjacent samples of the corresponding luminance component block.

37. The method according to article 28, characterized in that the availability of neighboring samples of the corresponding luminance component block is based on one or more of the following factors: the use of the coding mode of the current video block, the use of the coding mode of neighboring samples of the corresponding luminance component block, the use of a filter type associated with the neighboring samples of the corresponding block of the luminance component, or the position of adjacent samples of the corresponding block of the luminance component relative to the current video blocks or their subblocks.

38. The method according to Article 28, additionally containing:

in response to the lack of availability of neighboring samples from the corresponding block of the luminance component, substitution, replacement or filling of the unavailable samples with other samples.

39. The method according to Article 28, additionally containing:

applying a smoothing filter to samples adjacent to the corresponding block of the brightness component.

40. A method of video processing, containing:

performing a conversion between the current video block and a data bitstream representation of the current video block, where, in the conversion process, a second set of color component values of the current video block is obtained from a first set of color component values included in one or more reference frames, wherein the first set of color component values can be used in a linear model of the video encoding stage; And

in response to determining that a first set of chroma values included in one or more reference frames is a colocated luminance component block relative to the current video block, determining a second set of chroma values for the current video block is selectively activated or deactivated based on one or more conditions associated with a colocated luminance component block relative to the current video block.

41. The method according to Article 40, characterized in that the set of said one or more conditions associated with a luminance component block located at the same location relative to the current video block contains: the size of the partition units of the luminance component block located at the same location, reaching the number of coding units the luminance component block located at the same location reaches the threshold number, the upper left luminance component sample of the luminance component block located at the same location reaches the threshold size, the depth of the partition tree of the luminance component block located at the same location, or the corresponding luminance component block covers the upper left luminance component block component of a luminance component block located at the same location, and is further within a bounding box of a predetermined size.

42. The method according to article 40, characterized in that information indicating activation or deactivation of said second set definition is included in the data bit stream representation.

43. The method according to article 28, characterized in that the availability of neighboring samples of the corresponding block of the luminance component is associated with checking these neighboring samples in accordance with a predetermined order.

44. A method according to any one or more of clauses 1-43, characterized in that a second set of color component values for the current video block is stored for use in conjunction with one or more other video blocks.

45. A method according to any one or more of clauses 1 to 44, wherein said linear model corresponds to a cross-component linear model (CCLM) and said video encoding step corresponds to a luma chrominance scaling (LMCS) display mode.

46. A method according to any one or more of Clauses 1 to 45, wherein the current video block is a block encoded in an inter-prediction mode, a bidirectional mode, or an intra-block copy (IBC) mode.

47. A method according to any one or more of clauses 1-46, characterized in that the first set of color component values correspond to luminance component sample values, and the second set of color component values correspond to scaling factors for the chroma component.

48. A device in a video system containing a processor and a non-volatile storage device with commands recorded in it, when executed by the processor, this processor implements the method in accordance with any one of Articles 1 to 47.

49. A computer program product stored on computer-readable non-volatile storage media, this computer program product contains program code for implementing a method in accordance with any one of Articles 1 – 47.

In FIG. 12 is a block diagram showing an example of a video processing system 1200 in which the various methods described herein may be implemented. Various implementations may include some or all of the components of the system 1200. The system 1200 may include an input 1202 for receiving video content. This video content may be received in a native or uncompressed format, such as 8 or 10-bit multi-pixel pixel values, or may be in a compressed or encoded format. This input 1202 may be a network interface, a peripheral bus interface, or a storage device interface. Examples of network interfaces include wired bus interfaces such as Ethernet, passive optical network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

System 1200 may include an encoding component 1204 capable of implementing various encoding techniques described herein. The encoding component 1204 may reduce the average bit rate of video data from the input 1202 to the output of the encoding component 1204 to obtain an encoded representation of the video. Therefore, these encoding methods are sometimes called video compression methods or video transcoding methods. The output of encoding component 1204 may be either stored or transmitted via attached communications media, represented here by component 1206. The stored or transmitted (or encoded) representation of the video as a stream of data bits received as input 1202 may be used by component 1208 to generate values pixels or display video transmitted to display interface 1210. The procedure of generating user-viewable video from its representation as a stream of data bits is sometimes called video decompression (expansion). In addition, although some video processing operations are referred to as "encoding" operations or tools, it should be understood that the encoding tools or operations are used in the encoding devices and the corresponding decoding tools or operations that reverse the encoding results will be performed by the decoding device.

Examples of a peripheral bus interface or display interface may be a universal serial bus (USB) or a high definition multimedia interface (HDMI) or Displayport, etc. Examples of storage interfaces include the serial advanced technology attachment (SATA) interface, the peripheral device interface (PCI), the IDE interface, and other similar interfaces. The methods described herein may be implemented in a variety of electronic devices, such as mobile phones, laptop computers, smartphones, or other devices capable of digital processing and/or displaying video.

In FIG. 13 is a logic diagram of an example of a method for processing visual media signals. The steps of this logic circuit are discussed in conjunction with example options 7d and 7e8 in Section 4 of this document. At step 1302, the procedure performs a conversion between the current chroma video block of the visual media and a representation of that current chroma video block as a stream of data bits, where, during the conversion process, the remaining chrominance for the current chroma video block is scaled based on some scaling factor, where this the scaling factor is determined at least based on the luminance component samples located at predetermined positions.

In FIG. 14 is a logic diagram of an example of a method for processing visual media signals. The steps of this logic diagram are discussed in conjunction with the Case 1 example in Section 4 of this document. At step 1402, the routine converts between the current video block of visual media and a data bitstream representation of that current video block, where, in the conversion process, a second set of color component values for the current video block is determined using a cross-component linear model (CCLM) and /or in luma display mode with chrominance scaling (LMCS), from the first set of chroma values for the visual media.

In FIG. 15 is a logic diagram of an example of a method for processing visual media signals. The steps of this logic diagram are discussed in conjunction with the example of option 7 in Section 4 of this document. At step 1502, the routine converts between the current visual media video block and a data bitstream representation of that current video block, where, during the conversion process, one or more reconstructed samples associated with the current visual media frame are used to determine the scaling factor of the remaining chrominance component. at the processing stage in the luminance component display mode with chrominance component scaling (LMCS).

In FIG. 16 is a logic diagram of an example of a method for processing visual media signals. The steps of this logic circuit are discussed in conjunction with the example of option 8 in Section 4 of this document. At step 1602, the routine converts between the current video block of visual media and a representation of that current video block as a stream of data bits where, during the conversion process, one or more predicted luminance samples or reconstructed luminance samples are located in a current frame other than the reference frame. , and are used to determine the scaling factor of the remaining chrominance component at the processing stage in the luma display mode with chroma component scaling (LMCS).

In FIG. 17 is a logic diagram of an example of a method for processing visual media signals. The steps of this logic circuit are discussed in conjunction with example options 11a, 11b, 11c and 11d in Section 4 of this document. At step 1702, the procedure checks, in the process of converting between the current chrominance video block and the data bit stream representation of that current chroma video block, the availability of one or more adjacent luminance blocks from the corresponding luminance block that covers the upper left sample located therein. the same location of the brightness component block. At step 1704, the routine determines, based on the availability of one or more adjacent luminance blocks, whether adjacent luminance samples should be extracted from the corresponding luminance block. At step 1706, the procedure obtains, based on the specified determination, a scaling factor. At step 1708, the procedure scales, based on the found scaling factor, the chrominance residual for the current chrominance video block to generate a scaled chrominance residual. At step 1710, the procedure performs a conversion based on the scaled chrominance component residual.

In FIG. 18 is a logic diagram of an example of a method for processing visual media signals. The steps of this logic circuit are discussed in conjunction with the example of option 11 in Section 4 of this document. At step 1802, the procedure determines, in the process of converting between the current video block of visual media data and the data bitstream representation of that current video block, a second set of color component values for the current video block from the first set of color component values for the visual media data using a model associated with the step procedures wherein the first set of chroma values are adjacent samples from a corresponding luminance block covering the top left sample of a co-located luminance block.

In FIG. 19 is a logic diagram of an example of a method for processing visual media signals. The steps of this logic circuit are discussed in conjunction with the example of option 12 in Section 4 of this document. At step 1902, the procedure, in the process of converting between the current chrominance video block of the visual media and the data bitstream representation of that current chrominance video block, determines whether to selectively activate or deactivate the application of a cross-component linear model (CCLM) and/or residual scaling. chrominance component (CRS) for the current chrominance component video block, based at least in part on one or more conditions associated with a colocated luminance component block relative to the current chrominance component video block.

In FIG. 20 is a logic diagram of an example method for encoding visual media signals. The steps of this logic circuit are discussed in conjunction with the example of option 14 in Section 4 of this document. At step 2002, the procedure selectively enables or disables the application of luminance-dependent scaling of a chrominance residual (CRS) to the chrominance component of the current video block of visual media to encode the current video block into the video domain of the visual media and turn into a representation of the visual media as a stream of data bits. At step 2004, the procedure determines the inclusion or exclusion of a certain field from the representation of the visual media data in the form of a stream of data bits, where this field indicates selective activation or deactivation and, if included, it is signaled at a level other than the first syntax level. associated with the current video block.

In FIG. 21 is a logic diagram of an example method for decoding visual media signals. The steps of this logic circuit are discussed in conjunction with the example of option 14 in Section 4 of this document. At step 2102, the routine parses to extract a field from the data bitstream representation of the visual media where the field is included at a level other than the first syntax level associated with the current video block. At step 2104, the routine selectively enables or disables, based on this field, the application of luminance-dependent scaling of a chrominance residual (CRS) to the chrominance component of the current video block of visual media to generate a decoded video region based on the data bitstream representation.

In FIG. 22 is a logic diagram of an example method for encoding visual media signals. The steps of this logic diagram are discussed in conjunction with the example of option 15 in Section 4 of this document. At step 2202, the procedure selectively enables or disables the application of a cross-component linear model (CCLM) to the current visual media video block to encode the current video block into a representation of the visual media data as a stream of data bits. At step 2204, the procedure determines the inclusion or exclusion of a certain field from the representation of the visual media data in the form of a stream of data bits, where the field indicates selective activation or deactivation and, if included, it is signaled at a level other than the first syntax level. associated with the current video block.

In FIG. 23 is a logic diagram of an example method for decoding visual media signals. The steps of this logic diagram are discussed in conjunction with the example of option 15 in Section 4 of this document. At step 2302, the routine parses to extract a field from the data bitstream representation of the visual media where the field is included at a level other than the first syntax level associated with the current video block. At step 2304, the routine selectively enables or disables, based on this field, applying a cross-component linear model (CCLM) to the current video block of visual media to generate a decoded video region based on the data bitstream representation.

Some of the options discussed in this document are now presented in clause-by-clause format.

X1. A method for processing visual media data, comprising:

performing a conversion between a current chrominance video block of the visual media and a representation of that current chrominance video block as a stream of data bits, where, during the conversion process, the remainder of the chrominance component of the current chrominance video block is scaled based on a certain scaling factor, where this scaling factor is determined by at least based on samples of the brightness component located at specified positions.

X2. The method according to Article X1, characterized in that the specified scaling factor is calculated using a function applied to the luminance component samples located at specified positions.

X3. The method according to Article X1, characterized in that the function is a median value function or a rounding-based average value function.

X4. The method according to Article X1, characterized in that said predetermined positions are determined based on a luminance component block located at the same location corresponding to the current chrominance component video block.

A1. A method for processing visual media data, comprising:

performing a transformation between a current video block of visual media and a representation of that current video block as a stream of data bits, where, during the transformation process, a second set of color component values for the current video block is determined using a cross-component linear model (CCLM) and/or a procedure step in luma chroma scaling (LMCS) display mode, from the first set of chroma values for the visual media.

A2. The method according to Article A1, wherein said first set of chroma values represents reference samples of the current video block, and said second set of chroma values contains a scaling factor for the remainder of the chroma in LMCS mode.

A3. The method according to Article A2, characterized in that said reference samples are reference samples of the luminance component, subject to interpolation to determine the scaling factor of the remainder of the chrominance component.

A4. The method according to Article A1, characterized in that the first set of color component values are reference samples included in different reference frames.

A5. The method according to any one or more of articles A1 to A4, characterized in that the position of the reference sample is calculated based on the position of the corresponding luminance component value in the current video block and motion information of this current video block.

A6. The method according to Article A5, characterized in that the specified position of the corresponding luminance component value is the position of the upper left sample, the central sample or the lower right sample in the current video block.

A7. The method according to Article A6, characterized in that the motion information of the current video block corresponds to an integer motion vector or a fractional motion vector.

A8. The method according to Article A7, characterized in that the fractional motion vector is obtained using the fractional value of the luminance component in the reference frame.

A9. The method according to Article A7, characterized in that the integer motion vector is determined by rounding towards zero or away from zero.

A10. The method according to any one or more of articles A1-A2, characterized in that the first set of color component values are included in predefined reference frames of the visual media data.

A11. A method according to any one or more of articles A1-A10, characterized in that the median or average value of the first set of color component values is used to determine the second set of color component values for the current video block.

A12. The method according to article A10, characterized in that the plurality of predefined reference frames contains a frame with a reference index of the reference picture list.

A13. The method according to Article A12, characterized in that the reference index is zero and the list of reference images is zero.

A14. A method according to any one or more of Clauses A1 to A2, wherein a first set of color component values is included in multiple reference frames of the visual media and a weighted combination of the first set of color component values is used to determine a second set of color component values.

A15. A method according to Article A13, characterized in that the reference index and/or the list of reference pictures are signaled as fields in a data bit stream representation associated with one or more of the following objects: a sequence, a group of pictures, an image, a tile, a group of tiles, a slice, a sub-image, a string of coding tree units, a coding tree unit, a virtual pipeline data unit (VPDU), or a video block.

A16. The method of Clause A1, wherein a second set of color component values for the current video block is selectively determined from the first set of color component values based on whether the current video block is bidirectionally predicted.

A17. The method according to clause A16, characterized in that the second set of color component values for the current video block is individually determined for each prediction direction associated with the first set of color component values.

A18. The method of Clause A1, wherein a second set of color component values for the current video block is selectively determined from the first set of color component values based on whether the current video block is associated with subblock-based prediction.

A19. The method according to Article A18, characterized in that said sub-block-based prediction corresponds to affine prediction or alternative time motion vector prediction (ATMVP).

A20. A method according to any one or more of clauses A18-A19, characterized in that a second set of color component values for the current video block is determined for individual subblocks.

A21. The method according to any one or more of articles A18-A19, characterized in that the second set of color component values for the current video block is determined based on the entirety of the current video block, regardless of the subblock-based prediction.

A22. The method according to any one or more of articles A18-A21, characterized in that the first set of color component values is selected based at least in part on a motion vector of a subblock from the current video block.

A23. A method according to any one or more of clauses A18-A21, characterized in that a motion vector associated with a sub-block or a current video block is used to select a first set of color component values.

A24. A method according to any one or more of Articles A1 to A23, characterized in that the first set of color component values are intermediate color component values.

A25. A method according to any one or more of Articles A1 to A24, characterized in that a processing step in the LMCS scaling mode precedes another subsequent processing step.

A26. The method of Article A25, characterized in that the first set of color component values is selected based at least in part on an intermediate motion vector for a current video block or subblock of that current video block, and characterized in that the intermediate motion vector is calculated before another video encoding step.

A27. A method according to Article A26, wherein said other step comprises one or a combination of the following steps: a bidirectional optical flow (BDOF) step, a decoder-side motion vector refinement (DMVR) step, or a predictive optical flow refinement (PROF) step.

A28. A method for processing visual media data, comprising:

performing a conversion between the current video block of visual media and a representation of that current video block as a stream of data bits, where, during the conversion, one or more reconstructed samples associated with the current frame of visual media are used to determine the scaling factor of the remainder of the chrominance component during the mode processing step luma display with chrominance scaling (LMCS).

A29. The method according to article A28, characterized in that the current video block is encoded in intra-block copy (IBC) mode.

A30. The method according to article A28, characterized in that said one or more reconstructed samples are reference samples in a reference block associated with the current frame.

A31. Method according to Article A28, characterized in that the positions of one or more reconstructed samples are preset.

A32. The method according to Article A31, characterized in that said one or more reconstructed samples are reconstructed luminance samples located in a row at the top and a column at the left adjacent to a block covering the corresponding luminance component block relative to the current video block.

A33. The method of Clause A28, wherein the positions of the one or more reconstructed samples are based on the position of the corresponding luminance component block relative to the current video block and motion information of the current video block.

A34. A method for processing visual media data, comprising:

performing a conversion between a current video block of visual media and a representation of that current video block as a stream of data bits where, during the conversion process, one or more predicted luminance samples or reconstructed luminance samples are located in a current frame other than a reference frame and are used to determine scaling factor of the remaining chrominance component at the procedure stage in the luminance component display mode with chrominance component scaling (LMCS).

A35. The method according to Article A34, characterized in that one or more predicted luminance samples or reconstructed luminance samples are located in an adjacent MxN luminance block region covering the corresponding luminance block.

A36. The method according to article A35, characterized in that the corresponding luminance component block is a luminance component block located at the same location relative to the current video block.

A37. The method according to Article A36, characterized in that the product of M by N is less than the product of the block width by the block height for the luminance component block located in the same location relative to the current video block.

A38. The method according to Article A36, characterized in that M and N denote, respectively, a predetermined width and a predetermined height of a video block covering a luminance component block located at the same location relative to the current video block.

A39. The method according to Article A36, wherein M and N denote, respectively, a predetermined width and a predetermined height of a virtual pipeline data unit (VPDU) covering a colocated luminance component block relative to the current video block.

A40. A method according to any one or more of Articles A1 to A39, characterized in that, during the conversion process, the reference samples are used directly or downsampled before being used for determination.

A41. A method according to any one or more of articles A1-A40, characterized in that the samples used in determining the chroma residual scaling factors are selected based at least in part on one or more of the following information regarding the current video block: a quantization parameter , encoding mode or picture order number (POC).

A42. The method according to any one or more of Articles A1 to A41, wherein the current video block is an inter-prediction mode encoded block, an intra-prediction mode encoded block, a bidirectional prediction mode encoded block, or a block encoded in the bidirectional mode. Intrablock coding (IBC) mode.

A43. A method according to any one or more of clauses A1 to A42, characterized in that the first set of chroma component values correspond to luminance component sample values, and the second set of chroma component values correspond to scaling factors for the chrominance component of the current video block.

A44. A method according to any one or more of Articles A1 to A43, wherein the conversion procedure comprises generating a data bit stream representation based on the current video block.

A45. A method according to any one or more of Articles A1 to A43, wherein the conversion procedure comprises generating pixel values of the current video block based on a data bit stream representation.

A46. A video encoder comprising a processor configured to implement a method according to one or more of Articles A1 to A45.

A47. A video decoding device comprising a processor configured to implement a method according to one or more of Articles A1 to A45.

A48. A computer-readable storage medium with code written thereon, the code containing instructions executable by a processor to carry out the method of one or more of Clauses A1 to A38.

Y1. A method for processing visual media data, comprising:

checking, in the process of conversion between the current chrominance video block and the data bit stream representation of the current chrominance video block, the availability of one or more adjacent luminance blocks relative to a corresponding luminance block covering the upper left sample of a co-located luminance block;

determining, based on the availability of one or more adjacent luminance component blocks, whether adjacent luminance component samples relative to the corresponding luminance component block should be extracted;

obtaining a scaling factor based on said definition;

scaling, based on the obtained scaling factor, the chrominance component residual of the current chrominance component video block to generate a scaled chrominance component residual; And

performing a transformation based on the scaled remainder of the chrominance component.

Y2. The method according to article Y1, characterized in that the set of said one or more adjacent luminance component blocks contains a left-adjacent luminance component block and a top-adjacent luminance component block.

Y3. The method according to article Y1, characterized in that the set of said one or more adjacent samples of the brightness component contains one or more adjacent columns of samples to the left and/or one or more rows of samples adjacent to the top relative to the corresponding block of the brightness component.

Y4. A method according to any one or more of clauses Y1 or Y3, wherein said adjacent luminance samples are extracted and averaged based on rounding to obtain a scaling factor when one or more adjacent luminance blocks are available.

Y5. The method of Article Y1 or Y3, characterized in that it extracts said adjacent luminance samples and uses the median value of these adjacent luminance samples to determine a scaling factor in the case where one or more adjacent luminance blocks are available.

Y6. The method according to article Y3, characterized in that the number of adjacent luminance component samples is N, where 1<=N<=2W+2H, W and H denote the width and height of the corresponding luminance component block.

Y7. The method of clause Y1 or Y2, wherein the availability of one or more adjacent luminance component blocks is determined based on the width and/or height of the current image, tile, group of tiles, virtual pipeline data unit (VPDU), or slice.

Y8. The method of Clause Y7, wherein one or more adjacent luminance component blocks are unavailable in the case where the one or more adjacent blocks are located in another image, another tile, another group of tiles, another VPDU, or another slice.

Y9. The method according to article Y1, characterized in that adjacent luminance component samples relative to the corresponding block are skipped and not retrieved in the case where one or more adjacent luminance component blocks are not available.

Y10. The method according to article Y9, characterized in that the specified scaling factor is determined in the case where adjacent luminance component samples relative to the corresponding block are skipped and not extracted.

Y11. The method according to article Y10, characterized in that said scaling factor is determined based on a default value.

Y12. The method of clause Y11, wherein said default value is based on the bit depth of the current chrominance video block and a colocated luminance block.

Y13. The method according to clause Y12, wherein said default value is expressed as 1 << ( bitDepth − 1 ), where bitDepth denotes the bit depth of the current chroma video block and a colocated luminance block.

Y14. The method according to article Y1, characterized in that adjacent luminance component samples are reconstructed based on direct mapping.

B1. A method for processing visual media data, comprising:

determining, in the process of converting between the current video block of visual media data and the data bitstream representation of that current video block, a second set of color component values of the current video block from the first set of color component values of the visual media data using a model associated with a processing step, where the first set of color component values components are adjacent samples relative to the corresponding luminance component block covering the upper left sample of the co-located luminance component block.

B2. The method according to Article B1, characterized in that the current video block is one of: an intra-prediction encoded video block having a dual tree partition, or an intra-prediction encoded video block having a single tree partition encoded in an inter-prediction mode a video block that is partitioned in a single tree structure.

B3. A method according to any one or more of clauses B1-B2, characterized in that the first set of chroma component values corresponds to at least a portion of the reference samples identified at positions external to the corresponding luminance component block.

B4. The method according to article B3, characterized in that a portion of the reference samples identified in positions external to the corresponding luminance component block contains samples adjacent to the corresponding luminance component coding block.

B5. The method according to article B4, characterized in that the set of samples adjacent to the corresponding luminance component coding block contains N samples located in adjacent columns to the left and/or adjacent rows above relative to the corresponding luminance component coding block, where N=1...2W+2H , W and H denote the width and height of the corresponding luminance coding block.

B6. The method according to article B5, characterized in that the luminance component reading adjacent to the top is located at a point with coordinates (xCb + W/2, yCb - 1) or (xCb -1, yCb - 1), when the upper left reading located in the same place the brightness component block is located at a point with coordinates (xCb, yCb).

B7. The method according to article B5, characterized in that the adjacent left reading of the luminance component is located at a point with coordinates (xCb + W-1, yCb - 1), when the upper left reading of a block of luminance component located in the same place is located at a point with coordinates (xCb , yCb).

B8. The method according to article B4, characterized in that a portion of the reference samples identified at positions external to the corresponding luminance component block are located at predetermined positions.

B9. The method according to article B4, characterized in that the second set of color component values for the current video block is determined based on the median value or arithmetic mean value of N samples located in the left-adjacent columns and/or in the top-adjacent rows of the corresponding luma coding block.

B10. A method according to any one or more of clauses B1-B2, characterized in that a second set of chroma component values for the current video block is selectively determined from the first set of chroma component values based on the availability of adjacent samples relative to the corresponding luminance component block.

B11. The method of Clause B10, wherein the availability of adjacent samples of a corresponding luminance block is based on one or more of the following: the use of a coding mode for the current video block, the use of a coding mode for adjacent samples of the corresponding luminance block, the use of a filter type associated with neighboring samples of the corresponding luminance block, the location of adjacent samples of the corresponding luminance block relative to the current video blocks or subblocks of these blocks, the width of the current image/sub-image/tile/group of tiles/VPDU/slice, and/or the height of the current image/sub-image/tile/group of tiles /VPDUs/slice/string coding tree units (CTUs).

B12. The method according to Article B10, further comprising:

in response to determining the lack of availability of neighboring samples relative to the corresponding block of the luminance component, substituting, replacing or filling the unavailable samples with other samples.

B13. The method according to article B12, characterized in that the lack of availability of adjacent samples of the corresponding luminance component block is based at least in part on determining when the encoding mode of the current video block is an inter-frame prediction mode, the adjacent samples encoding mode is an intra-frame prediction mode and/or an intra-frame block copy mode (IBC) and/or combined inter-intra prediction (CIIP) mode and/or local light compensation (LIC) mode.

B14. The method according to article B12, characterized in that the lack of availability of neighboring samples of the corresponding luminance component block is based at least in part on the determination that when the coding mode of the current video block is an inter-frame prediction mode, the neighboring samples are subjected to diffuse filtering and/or bilateral filtering and/or filtering based on the Hadamard transform.

B15. The method according to article B12, characterized in that the lack of availability of neighboring samples of the corresponding luminance component block is based at least in part on the determination that the neighboring block is located outside the current picture/sub-picture/tile/tile group/VPDU/slice/line of coding tree units ( CTU) associated with the current video block.

B16. A method according to any one or more of Articles B12-B15, further comprising:

in response to determining a lack of availability of adjacent samples relative to the corresponding luminance component block, deactivating the determination of the second set of color component values relative to the current video block from the first set of color component values.

B17. The method according to Article B10, further comprising:

in response to determining that the number of available adjacent samples is less than a threshold, deactivating the determination of the second set of color component values relative to the current video block from the first set of color component values.

B18. Method according to Article B17, characterized in that said threshold value is equal to one.

B19. The method of Clause B12, wherein if an adjacent sample is determined to be unavailable, then the adjacent sample is replaced with 1<<( bitDepth −1) samples, where bitDepth denotes the bit depth of the first set of color component value samples or the second set of color component value samples.

B20. The method according to article B12, characterized in that if a neighbor sample is determined to be unavailable, then the first available neighbor sample is substituted for that neighbor sample in accordance with a predetermined check order.

B21. The method according to article B13, characterized in that if a neighbor sample is determined to be unavailable, then the neighbor sample is populated using one or more of the left neighbor sample, the right neighbor sample, the top neighbor sample, or the bottom neighbor sample.

B22. The method according to Article B10, further comprising:

applying an anti-aliasing filter to samples adjacent to the corresponding luma component block used in determining the second set of color component values for the current video block.

B23. The method of Article B22, wherein the anti-aliasing filter comprises one or more of the following: a two-way filter, a Hadamard transform filter, or a direct mapping of the reshaping region.

B24. A method according to any one or more of clauses B1-B23, characterized in that a second set of color component values for the current video block is stored for use in connection with one or more other video blocks.

B25. A method according to any one or more of articles B1-B23, characterized in that said model corresponds to a cross-component linear model (CCLM) and/or said processing step corresponds to a luminance component scaling (LMCS) display mode.

B26. The method according to any one or more of clauses B1 to B23, wherein the current video block is an intra-prediction mode encoded block, an inter-prediction mode encoded block, a bi-directional prediction mode encoded block, or a block encoded in the inter-prediction mode. in Intra Block Coding (IBC) mode.

B27. A method according to any one or more of articles B1-B23, characterized in that the first set of chroma component values corresponds to luminance component sample values, and the second set of chroma component values corresponds to chrominance component scaling factors for the current video block.

B28. A method according to any one or more of clauses B1 to B23, wherein the conversion procedure comprises generating a data bit stream representation based on the current video block.

B29. A method according to any one or more of articles B1-B23, characterized in that the conversion procedure comprises generating pixel values of the current video block based on its representation as a stream of data bits.

B30. A video encoder comprising a processor configured to implement a method according to one or more of clauses B1 to B23.

B31. A video decoding device comprising a processor configured to implement a method according to one or more of clauses B1 to B23.

B32. A computer-readable storage medium with code written thereon, the code containing instructions executable by a processor to implement a method according to one or more of clauses B1-B23.

C1. A method for processing visual media data, comprising:

in the process of converting between a current chrominance video block of visual media and a data bit stream representation of that current chroma video block, making a determination regarding selective activation or deactivation of the application of a cross-component linear model (CCLM) and/or chroma residual scaling (CRS) to the current chroma video block based at least in part on one or more conditions associated with a colocated luminance block from the current chrominance video block.

C2. The method according to Article C1, characterized in that the set of said one or more conditions associated with a co-located luminance component block for the current chroma video block contains: the size of the partition units of the co-located luminance component block, achieving the number of coding units from the luminance component block located at the same location reaches the threshold number, the upper left sample of the luminance component from the luminance component block located at the same location reaches the threshold size, the depth of the partition tree of the luminance component block located at the same location, or the corresponding luminance component block covers the upper left sample luminance component of the colocated luminance component block, and is further within a bounding box of a predetermined size, an encoding mode of one or more coding units (CUs) covering at least one sample of the colocated luminance component block, and/ or the dimensions of the current video block of the chrominance component.

C3. A method according to any one or more of clauses C1-C2, characterized in that the activation of application of a CCLM model and/or CRS scaling to a current chrominance component video block is canceled in response to determining that a co-located luminance component block for the current video block the color component is divided into several partitioning units.

C4. A method according to any one or more of Clauses C1-C2, wherein application of a CCLM model and/or CRS scaling to a current chrominance video block is activated in response to determining that the colocated luminance component block is relative to the current chroma video block component is not divided into several partition units.

C5. A method according to any one or more of Clauses C1-C2, characterized in that the activation of application of a CCLM model and/or CRS scaling to a current chrominance video block is canceled in response to determining that a colocated luminance component block is relative to the current video block chroma component contains more than one of the following objects: a threshold number of coding units and/or a threshold number of partitioning units and/or a threshold number of transform units.

C6. Method according to Article C5, characterized in that the threshold number is equal to one.

C7. The method of Clause C5, wherein the threshold number is based at least in part on whether a CCLM model and/or CRS scaling is applied.

C8. A method according to Article C5, characterized in that the threshold number is fixed or is represented as a stream of data bits.

C9. The method of Clause C5, wherein the threshold number is based at least in part on profiles/levels/tiers associated with the current chrominance video block.

C10. A method according to Article C5, characterized in that said coding units and/or partitioning units and/or transform units are entirely within a co-located luminance component block.

C11. A method according to Article C5, characterized in that said coding units and/or partitioning units and/or transform units are partially located within a co-located luminance component block.

C12. Method according to Article C11, characterized in that said coding units and/or partitioning units and/or transform units are partially located along the boundary of a co-located luminance component block.

C13. The method according to clause C5, characterized in that said coding units and/or partitioning units and/or transformation units are associated with subblock-based prediction.

C14. The method according to clause C13, characterized in that the subblock-based prediction corresponds to intra-subdivision (ISP) or affine prediction or alternative temporal motion vector prediction (ATMVP).

C15. A method according to any one or more of clauses C1-C2, characterized in that the activation of application of the CCLM model and/or CRS scaling to the current chrominance component video block is canceled in response to determining that the size of the coding unit and/or partitioning unit and/or a conversion unit covering the upper left luminance component sample of a colocated luminance component block greater than a specified block size.

C16. The method according to Article C15, characterized in that the luminance component block located at the same location has a size of 32x32 and is included in a corresponding luminance component block of size 64x64, and the predetermined luminance component block has a size of 32x64.

C17. The method of Clause C2, wherein the application of a CCLM model and/or CRS scaling to a current chrominance video block is activated in response to determining that a colocated luminance block with respect to the current chrominance video block has not been split, and the corresponding luminance block component covering the upper left luminance component sample of a colocated luminance component block is entirely within a bounding box of a given size.

C18. The method according to Article C17, characterized in that said corresponding luminance component block has a size of 32x64 and the bounding box has a size of 40x70.

C19. The method of Clause C17, wherein the predetermined size of the bounding box is based in part on the size of a coding tree unit (CTU) associated with the current chroma video block and/or on the size of a coding unit (CU) associated with the current chrominance video block.

C20. A method according to any one or more of Articles C1-C2, characterized in that a luminance block located at the same location relative to the current chrominance video block is divided into several partition units, and the predicted samples or reconstructed samples within these several partition units are used to determining values associated with the CRS scaling of the current chrominance component video block.

C21. The method of Clause C20, wherein the average of the predicted samples or reconstructed samples within the first partition unit of a co-located luminance block relative to the current chrominance video block is used to determine values associated with the CRS scaling of the current chrominance video block .

C22. The method of Clause C20, wherein a top left predicted sample or top left reconstructed sample within a first partition unit of a co-located luminance block relative to a current chroma video block is used to determine values associated with the CRS scaling of the current chroma video block.

C23. The method of Clause C20, wherein a central predicted sample or a central reconstructed sample within a first partition unit of a co-located luminance component block relative to a current chroma video block is used to determine the values of the color components of the current chrominance video block.

C24. The method according to Clause C2, characterized in that the activation of the application of the CCLM model and/or CRS scaling to the current chrominance component video block is canceled in response to determining that the coding mode of one or more coding units (CUs) covering at least one sample located in that at the same location of the luminance component block, represents one of the modes: affine mode, bidirectional prediction mode, bidirectional optical flow (BDOF) mode, DMVR refinement mode, affine matrix prediction mode, inter-frame prediction mode, or intra-frame subdivision (ISP) mode.

C25. The method according to Article C2, characterized in that one or more coding units (CUs) covering at least one sample of a co-located luminance component block constitute a corresponding luminance component block.

C26. A method according to any one or more of Articles C1 to C25, further comprising:

an indication, based on some field in the bitstream representation, that the CCLM model and/or CRS scaling is selectively activating or deactivating the current chrominance component video block.

C27. A method according to any one or more of clauses C1-C26, characterized in that selective activation or deactivation of the application of the CCLM model and/or CRS scaling for the current chrominance video block is carried out on one or more subblocks of the current chrominance video block.

C28. The method according to article C27, characterized in that said one or more subblocks from the current chrominance component video block have a size of 2x2 or 4x4.

C29. The method according to Article C27, characterized in that the application of the CCLM model and/or CRS scaling is activated for a subblock of the current chroma video block when the corresponding luminance coding block relative to the current chrominance video block covers all samples of the corresponding block for that subblock.

C30. The method according to clause C27, characterized in that the activation of the application of the CCLM model and/or CRS scaling for a subblock of the current chroma video block is canceled when all samples of the corresponding block for this subblock are not covered by the corresponding luminance coding block.

C31. The method according to clause C27, characterized in that the CCLM model and/or CRS scaling parameters are associated with each subblock of the current chroma video block.

C32. The method of Clause C27, wherein selectively enabling or disabling the application of a CCLM model and/or CRS scaling to a sub-block of a current chroma video block is based on samples included in a co-located luminance block.

C33. The method according to any one or more of Clauses C1 to C32, wherein the current chrominance video block is an inter-prediction mode encoded block, an intra-prediction mode encoded block, a bidirectional prediction mode encoded block, or a block , encoded in intra-block copy (IBC) mode.

C34. A method according to any one or more of Articles C1 to C33, wherein the conversion procedure comprises generating a data bitstream representation based on the current chrominance component video block.

C35. A method according to any one or more of Articles C1-C33, characterized in that the conversion procedure comprises generating pixel values of the current chrominance component video block based on a data bit stream representation.

C36. A video encoder comprising a processor configured to implement a method according to one or more of Articles C1 to C33.

C37. A video decoding device comprising a processor configured to implement a method according to one or more of Articles C1 to C33.

C38. A computer-readable storage medium with code written thereon, the code containing instructions executable by a processor to implement a method according to one or more of Clauses C1 to C33.

D1. A method for encoding visual media data, comprising:

selectively enabling or disabling the application of luminance-dependent chrominance residual scaling (CRS) to a chrominance component of the current video block of the visual media for the purpose of encoding the current video block in the video domain of the visual media and converting it into a representation of the visual media as a stream of data bits; And

deciding to include or exclude a certain field in the representation of visual media data as a stream of data bits, where this field indicates selective activation or deactivation and, if this field is included, signaling it at a level other than the first syntactic level associated with current video block.

D2. A method for decoding visual media data, comprising:

performing parsing to extract a certain field from the representation of visual media data as a stream of data bits, where the field is included at a level different from the first syntactic level associated with the current video block; And

selectively activating or deactivating, based on said field, applying luminance dependent chrominance residual scaling (CRS) to the chrominance component of the current video block of visual media data to generate a decoded video region based on the data bitstream representation.

D3. A method according to any one or more of clauses D1-D2, characterized in that the first syntax level is a tile group header level, and characterized in that said field is included in one of the objects: a sequence parameter set (SPS) associated with the current video block, a tile associated with the current video block, a string of coding tree units (CTU) associated with the current video block, a coding tree unit (CTU) associated with the current video block, a virtual pipeline data unit (VPDU) associated with the current video block, or a unit coding unit (CU) associated with the current video block.

D4. A method according to any one or more of clauses D1 to D3, characterized in that said field is a flag designated chroma_residual_scale_flag.

D5. A method according to any one or more of articles D1-D4, characterized in that said field is associated with one of the syntactic levels, and characterized in that if this field is equal to one, the application of luminance-dependent scaling of the remainder of the chroma component at the level is activated below the specified syntax level, and if this field is zero, the application of luma-dependent scaling of the remainder of the chrominance component at a level below the specified syntax level is not enabled.

D6. A method according to article D5, characterized in that the specified field is associated with the level of the split node, and characterized in that if this field is equal to one, the use of luminance component-dependent scaling of the remainder of the chrominance component at the level of the specified split node is activated, and if this field is equal to zero , the use of luminance component-dependent scaling of the remainder of the chrominance component at the level of the specified split node is not activated.

D7. A method according to any one or more of clauses D1-D4, characterized in that said field is associated with a threshold size, and characterized in that when said field is equal to one, the application of luminance-dependent scaling of the remaining chrominance component is enabled for video blocks having the specified threshold size or greater than this size, and if this field is zero, the application of luma-dependent scaling of the remaining chrominance component for video blocks smaller than the threshold size is not enabled.

D8. The method according to article D7, characterized in that the specified threshold size is 32x32.

D9. A method according to any one or more of clauses D1 to D8, characterized in that said field is not signaled in said bitstream representation and the absence of that field in said data bitstream representation is used to infer that the use of luminance component-dependent scaling of the remainder of the chrominance component is not activated, and this field is set to zero.

D10. A method according to any one or more of articles D1-D9, characterized in that values associated with luminance dependent CRS scaling for the current video block are stored for use in connection with one or more other video blocks.

D11. The method according to clause D10, characterized in that the values associated with the luminance dependent CRS scaling are determined after encoding or decoding the luminance component block.

D12. The method according to clause D11, characterized in that, in the luminance component block, predicted samples and/or intermediate predicted samples and/or reconstructed samples and/or reconstructed samples before contour filtering are used to determine values associated with luminance-dependent scaling of the CRS.

D13. Method according to article D12, characterized in that the loop filtering procedure comprises the use of: a deblocking filter and/or a nonlinear adaptive offset filter (SAO) and/or a two-way filter and/or a filter based on the Hadamard transform and/or an adaptive loop filter (ALF) .

D14. The method of Clause D11, wherein the samples in the bottom row and/or right column of the luminance block are used to determine values associated with the luminance-dependent scaling of the CRS.

D15. The method according to clause D11, characterized in that the samples associated with adjacent blocks are used to determine values associated with the luminance-dependent scaling of the CRS.

D16. The method according to clause D15, wherein the current video block is an intra-prediction mode encoded block, an inter-prediction mode encoded block, a bidirectional prediction mode encoded block, or an intra-block copy (IBC) encoded block.

D17. The method according to article D15, characterized in that the availability of samples associated with neighboring blocks is checked in accordance with a predetermined order.

D18. The method according to Article D17, characterized in that said predetermined order relative to the current video block is one of: from left to right, from top left towards top right, from left towards top, from top left towards top right, from top towards the left, from the top right towards the top left.

D19. The method according to Article D17, characterized in that said predetermined order relative to the current video block is one of: bottom left - left - top right - top - top left.

D20. The method according to article D17, characterized in that said preset order relative to the current video block is one of: left - top - top right - bottom left - top left.

D21. A method according to article D17, characterized in that said predetermined order is associated with samples in the first available subset of neighboring blocks.

D22. The method according to clause D15, characterized in that if the current video block is encoded in the inter-prediction mode and the adjacent block is a block encoded in the intra-prediction mode, a block encoded in the IBC copy mode, or a block encoded in the combined CIIP prediction mode, then the samples associated with the adjacent block are determined to be unavailable.

D23. The method according to clause D15, characterized in that if the current video block is a block encoded in a combined CIIP prediction mode and the adjacent block is a block encoded in an intra-frame prediction mode, a block encoded in an IBC copy mode, or a block encoded in a combined mode CIIP prediction, then the samples associated with the adjacent block are determined to be unavailable.

D24. The method according to Article D15, further comprising:

in response to determining that the number of neighboring blocks is less than a threshold, disabling the activation of the luminance-dependent scaling CRS determination.

D25. Method according to Article D24, characterized in that the threshold value is equal to one.

D26. The method according to clause D24, characterized in that if any sample from an adjacent block is determined to be unavailable, then this sample is replaced with samples 1 << ( bitDepth − 1), where bitDepth denotes the bit depth of the chrominance components or luma components.

E1. A method for encoding visual media data, comprising:

selectively enabling or disabling the application of a cross-component linear model (CCLM) to the current visual media video block to encode the current video block into a representation of the visual media data as a stream of data bits; And

making a determination to include or exclude a certain field from the representation of the visual media data as a bitstream of data, where the field indicates the specified selective activation or deactivation and, if included, it is signaled at a layer other than the first syntactic level associated with current video block.

E2. A method for decoding visual media data, comprising:

performing parsing to extract a certain field from a representation of the visual media data as a stream of data bits, where the field is included at a level different from the first syntactic level associated with the current video block; And

selectively activating or deactivating, based on the specified field, applying a cross-component linear model (CCLM) to the current video block of visual media data to generate a decoded video region based on the specified data bit stream representation.

E3. A method according to any one or more of clauses E1-E2, characterized in that the first syntactic level is a sequence parameter set (SPS) level, and characterized in that said field is included in one of the objects: a picture parameter set (PPS) , associated with the current video block, slice associated with the current video block, image title associated with the current video block, tile associated with the current video block, tile group associated with the current video block, line of coding tree units (CTU) associated with the current video block, unit a coding tree unit (CTU) associated with the current video block, a virtual pipeline data unit (VPDU) associated with the current video block, or a coding unit (CU) associated with the current video block.

E4. A method according to any one or more of clauses E1-E3, characterized in that said field is a flag designated cclm_flag.

E5. A method according to any one or more of clauses E1-E4, characterized in that the absence of the specified field in the data bitstream representation is used to recognize that the application of the CCLM model is not activated.

E6. A method according to any one or more of clauses E1-E4, characterized in that the presence of said field in the data bitstream representation is used to recognize that application of the CCLM model has been activated.

E7. The method of Clause E5, wherein if the size of the current video block is not greater than a threshold size, the specified field is excluded from the data bitstream representation, and thereby the exclusion of this field is used to recognize that the application of the CCLM model is not activated.

E8. Method according to Article E7, characterized in that the threshold size is 8x8.

F1. A video encoder comprising a processor configured to implement a method according to one or more of Articles X1-E8.

F2. A video decoding device comprising a processor configured to implement a method according to one or more of Articles X1-E8.

F3. A computer-readable storage medium with code written thereon, the code containing instructions executable by a processor to carry out a method according to one or more of clauses X1-E8.

As used herein, the term “video processing” or “visual media processing” may refer to video encoding, video decoding, video compression, or video expansion (decompression). For example, video compression algorithms may be applied in the process of converting from a pixel representation of a video to a corresponding bitstream representation or vice versa. The representation of the current video block as a data bitstream may, for example, correspond to bits that are either located in one location or distributed in different locations in the data bitstream, as defined in the syntax. For example, a macroblock may be encoded in terms of transformed and encoded error residuals, as well as the use of bits in headers and other fields in the data bitstream. In addition, during the conversion process, the decoding device can parse the data bit stream taking into account the knowledge that some fields may be present or absent based on the specified definition, as described in the above technical solutions. Likewise, the encoder may determine that certain syntactic fields should or should not be included in the encoded representation and generate one accordingly by including or excluding those syntactic fields from the encoded representation.

From the foregoing, it will be understood that specific embodiments of the technology proposed herein have been described for purposes of illustration, but various modifications may be made without departing from the scope of the present invention. Accordingly, the proposed technology is not limited by anything other than the attached Claims.

Embodiments of the subject matter of the present invention and the functional operations described herein may be implemented in a variety of systems, digital electronic circuits or computer downloadable software, firmware or hardware, including the structures discussed herein and their structural equivalents, or in a combination of one or more of the listed components. The embodiments of the subject matter of the present invention discussed in this description may be implemented in the form of one or more computer program products, i.e. one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable storage medium for execution by a data processing device or for controlling the operation of that device. Such a computer-readable storage medium may be a computer-readable storage device, a computer-readable storage board, a storage device, a composition of objects implementing a computer-readable propagated signal, or a combination of one or more of these components. The term "data processing unit" or "data processing device" covers all data processing hardware, devices, and machines, including, by way of example, a programmable processor, a computer, or multiple processors or computers. The hardware may contain, in addition to hardware, code constituting the environment for executing the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these components.

A computer program (also known as a program, software, software application, script or code) may be written in any programming language, including compiled or interpreted languages, and may be deployed in any form, including a stand-alone program, or in the form of a module, component, subroutine, or other unit suitable for use in a computer environment. A computer program does not necessarily correspond to a file on a file system. A program may be stored in a portion of a file that also contains other programs or data (for example, one or more scripts are stored in a markup language document), in a single file specifically dedicated only to the program in question, or in multiple coordinated files (for example, files that save one or more modules, subroutines or code fragments). A computer program may be deployed for execution on a single computer or on multiple computers located at a single location or distributed over multiple locations and connected through a communications network.

The procedures and logic described herein may be implemented by one or more programmable processors executing one or more computer programs to implement functions by operating on input data and generating output data. These procedures and logic circuits may also be implemented by, and the hardware may also be implemented in, the form of special-purpose logic circuitry, such as a field programmable gate array (FPGA) or application specific integrated circuit (ASIC). circuit)).

Processors suitable for executing a computer program include, for example, general purpose and special purpose microprocessors and any one or more digital computer processors of any type. In general, the processor will receive instructions and data from read only memory and/or random access memory. The main elements of a computer are a processor for executing commands and one or more storage devices for storing commands and data. In general, a computer will also include, or be operatively coupled to receive data and/or transmit data, one or more mass storage devices for storing data, such as magnetic devices, magneto-optical disks, or optical disks. However, the computer does not need to have such devices. Computer-readable media for storing computer program instructions and data includes all forms of non-volatile storage devices and storage media, including, for example, semiconductor storage devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable memory (EPROM). EEPROM) and flash memory devices. The processor and storage device can be augmented or embedded in special-purpose logic circuitry.

This description, together with the accompanying drawings, should be considered as examples only. As used here, the application of the conjunction "or" must also include "and/or" unless the context clearly indicates otherwise.

Although this patent document contains many specific details, they should not be construed as any limitation on the scope of any innovation or what may be claimed as an invention, but merely as descriptions of features that may be specific to particular embodiments of particular innovations . Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of one embodiment may also be implemented in multiple embodiments individually or in some suitable subcombination. More specifically, although features may be described above as operating in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be excluded from that claimed combination such that the claimed combination may be converted into a subcombination or variations of such a subcombination.

Likewise, while operations are depicted in the drawings in a particular order, this should not be construed as a requirement that such operations must be performed in the specific order shown or in a sequential order or that all illustrations shown must be performed to achieve the desired results. Moreover, the separation of the various system components described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few embodiments and examples are described herein, however, other embodiments, improvements and variations may be created based on what is described and illustrated herein.

Claims (47)

1. A method for processing video data, comprising the steps of: determining, for conversion between a video chrominance block and a data bitstream representation of the video, whether the intra-frame prediction mode using a cross-component linear model for the chrominance block is to be activated or deactivated, based on whether one or more luminance coding units are encoded components covering at least one sample of a corresponding luminance component region of a chroma component block, in a subblock-based prediction mode, wherein the chroma component block uses a dual partition tree structure, in which the luminance component and at least one chroma component have separate partition tree structures; in response to the intra-frame prediction mode using the cross-component linear model being activated, determining parameters of the cross-component linear model based on at least adjacent chroma samples relative to the chroma block; apply a cross-component linear model to obtain predicted values for the chrominance component block based on the specified parameters; And perform the specified transformation based on the specified predicted values, in this case, the parameters of the cross-component linear model are obtained by performing the steps in which: checking the availability of one or more adjacent chrominance component blocks relative to the specified chrominance component block; find R chrominance component samples from adjacent chrominance component samples of said one or more adjacent chrominance component blocks to determine a set of parameter values for the cross-component linear model, where R is an integer greater than 2, wherein the at least one adjacent chrominance sample does not belong to said R chrominance samples based on the chrominance block size. 2. The method according to claim 1, wherein the subblock-based prediction mode corresponds to the intra-frame subdivision mode. 3. The method of claim 1, wherein the cross-component linear model is deactivated in response to said one or more luminance component coding units being encoded in a subblock-based prediction mode. 4. The method of claim 1, wherein determining whether to activate or deactivate the intra-frame prediction mode using a cross-component linear model for a chroma block is further based on partitioning information of said one or more luminance coding units. 5. The method of claim 4, wherein the partition information comprises an indication of the depth of the partition tree for said one or more luma coding units. 6. The method of claim 5, wherein the depth of the partition tree is the depth of the quad tree. 7. The method of claim 1, wherein determining whether to enable or disable the intra-frame prediction mode using a cross-component linear model for a chrominance block is further based on the block size of one or more chrominance coding units covering at least one counting the corresponding region of the chrominance component of the specified block of the chrominance component. 8. The method of claim 1, wherein determining whether the intra-frame prediction mode using the cross-component linear model for a chrominance block is to be activated or deactivated is further based on whether said one or more luminance coding units are split by an operation on a scheme of multiple types of partition trees, different from the operation of a quadruple partition tree scheme. 9. The method of claim 1, wherein in response to said one or more adjacent chrominance blocks being unavailable, the predicted values of said chrominance block are set according to the formula 1 << (bitDepth − 1 ), where bitDepth is the bit depth video. 10. The method according to claim 1, in which, during the specified transformation, a block of the chrominance component is encoded into a stream of data bits. 11. The method according to claim 1, in which, during the specified transformation, a block of the chrominance component is decoded from the data bit stream. 12. A video data processing device comprising a processor and a non-volatile storage device with commands written therein, wherein the commands, when executed by the processor, cause the processor to execute: determining, for conversion between a chrominance block of a video and a data bitstream representation of the video, whether the intra-frame prediction mode using a cross-component linear model should be activated or deactivated for the chrominance block, based on whether one or more luminance coding units are encoded components covering at least one sample of a corresponding luminance component region of a chroma component block, in a subblock-based prediction mode, wherein the chroma component block uses a dual partition tree structure, in which the luminance component and at least one chroma component have separate partition tree structures; in response to the intra-frame prediction mode using the cross-component linear model being activated, determining parameters of the cross-component linear model based on at least adjacent chrominance samples relative to the chroma block; applying a cross-component linear model to obtain predicted values for a block of the chrominance component based on the specified parameters; And performing the specified transformation based on the specified predicted values, in this case, the parameters of the cross-component linear model are obtained by: checking the availability of one or more adjacent chrominance blocks relative to the specified chrominance block; finding R samples of the chrominance component from adjacent samples of the chrominance component of the specified one or more adjacent blocks of the chrominance component to determine a set of values for the parameters of the cross-component linear model, where R is an integer greater than 2, wherein the at least one adjacent chrominance sample does not belong to said R chrominance samples based on the chrominance block size. 13. A non-volatile computer-readable storage medium that stores instructions that cause the processor to execute: determining, for conversion between a chrominance block of a video and a data bitstream representation of the video, whether the intra-frame prediction mode using a cross-component linear model should be activated or deactivated for the chrominance block, based on whether one or more luminance coding units are encoded components covering at least one sample of a corresponding luminance component region of a chroma component block, in a subblock-based prediction mode, wherein the chroma component block uses a dual partition tree structure, in which the luminance component and at least one chroma component have separate partition tree structures; in response to the intra-frame prediction mode using the cross-component linear model being activated, determining parameters of the cross-component linear model based on at least adjacent chroma samples relative to the chroma block; applying a cross-component linear model to obtain predicted values for a block of the chrominance component based on the specified parameters; And performing the specified transformation based on the specified predicted values, in this case, the parameters of the cross-component linear model are obtained by: checking the availability of one or more adjacent chrominance blocks relative to the specified chrominance block; finding R samples of the chrominance component from adjacent samples of the chrominance component of the specified one or more adjacent blocks of the chrominance component to determine a set of values for the parameters of the cross-component linear model, where R is an integer greater than 2, wherein the at least one adjacent chrominance sample does not belong to said R chrominance samples based on the chrominance block size. 14. A method for storing a video data bit stream, comprising the steps of: determining whether to activate or deactivate the intra-frame prediction mode using a cross-component linear model for a chroma block based on whether one or more luminance coding units covering at least one sample of a corresponding luminance region of the chrominance block are encoded, in a subblock-based prediction mode, wherein the chrominance component block uses a dual split tree structure, in which the luma component and at least one chrominance component have separate split tree structures; in response to the intra-frame prediction mode using the cross-component linear model being activated, determining parameters of the cross-component linear model based on at least adjacent chroma samples relative to the chroma block; apply a cross-component linear model to obtain predicted values for the chrominance component block based on the specified parameters; generating a stream of video data bits based on said predicted values; And storing a stream of data bits on a non-volatile computer-readable medium for recording information, in this case, the parameters of the cross-component linear model are obtained by performing the steps in which: checking the availability of one or more adjacent chrominance component blocks relative to the specified chrominance component block; find R chrominance component samples from adjacent chrominance component samples of said one or more adjacent chrominance component blocks to determine a set of parameter values for the cross-component linear model, where R is an integer greater than 2, wherein the at least one adjacent chrominance sample does not belong to said R chrominance samples based on the chrominance block size.