[go: up one dir, main page]

WO2024153220A9 - Procédés et appareil de génération d'échantillon de limite pour une copie intra-bloc et mise en correspondance intra-modèle dans un codage vidéo - Google Patents

Procédés et appareil de génération d'échantillon de limite pour une copie intra-bloc et mise en correspondance intra-modèle dans un codage vidéo

Info

Publication number
WO2024153220A9
WO2024153220A9 PCT/CN2024/073213 CN2024073213W WO2024153220A9 WO 2024153220 A9 WO2024153220 A9 WO 2024153220A9 CN 2024073213 W CN2024073213 W CN 2024073213W WO 2024153220 A9 WO2024153220 A9 WO 2024153220A9
Authority
WO
WIPO (PCT)
Prior art keywords
block
samples
padding
boundary
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/073213
Other languages
English (en)
Other versions
WO2024153220A1 (fr
Inventor
Chen-Yen LAI
Tzu-Der Chuang
Chih-Wei Hsu
Ching-Yeh Chen
Yu-Wen Huang
Yi-Wen Chen
Yu-Cheng Lin
Chih-Hsuan Lo
Man-Shu CHIANG
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
MediaTek Inc
Original Assignee
MediaTek Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by MediaTek Inc filed Critical MediaTek Inc
Priority to EP24744390.6A priority Critical patent/EP4652739A1/fr
Publication of WO2024153220A1 publication Critical patent/WO2024153220A1/fr
Anticipated expiration legal-status Critical
Publication of WO2024153220A9 publication Critical patent/WO2024153220A9/fr
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/563—Motion estimation with padding, i.e. with filling of non-object values in an arbitrarily shaped picture block or region for estimation purposes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding

Definitions

  • the present invention is a non-Provisional Application of and claims priority to U.S. Provisional Patent Application No. 63/480,551, filed on January 19, 2023, U.S. Provisional Patent Application No. 63/494,498, filed on April 6, 2023 and U.S. Provisional Patent Application No. 63/512,312, filed on July 7, 2023.
  • the U.S. Provisional Patent Applications are hereby incorporated by reference in their entireties.
  • the present invention relates to video coding system.
  • the present invention relates to padding sample generation for out of picture region and chroma coding using direct block vector in video coding systems.
  • VVC Versatile video coding
  • JVET Joint Video Experts Team
  • MPEG ISO/IEC Moving Picture Experts Group
  • ISO/IEC 23090-3 2021
  • Information technology -Coded representation of immersive media -Part 3 Versatile video coding, published Feb. 2021.
  • VVC is developed based on its predecessor HEVC (High Efficiency Video Coding) by adding more coding tools to improve coding efficiency and also to handle various types of video sources including 3-dimensional (3D) video signals.
  • HEVC High Efficiency Video Coding
  • Fig. 1A illustrates an exemplary adaptive Inter/Intra video encoding system incorporating loop processing.
  • Intra Prediction 110 the prediction data is derived based on previously coded video data in the current picture.
  • Motion Estimation (ME) is performed at the encoder side and Motion Compensation (MC) is performed based on the result of ME to provide prediction data derived from other picture (s) and motion data.
  • Switch 114 selects Intra Prediction 110 or Inter-Prediction 112 and the selected prediction data is supplied to Adder 116 to form prediction errors, also called residues.
  • the prediction error is then processed by Transform (T) 118 followed by Quantization (Q) 120.
  • T Transform
  • Q Quantization
  • the transformed and quantized residues are then coded by Entropy Encoder 122 to be included in a video bitstream corresponding to the compressed video data.
  • the bitstream associated with the transform coefficients is then packed with side information such as motion and coding modes associated with Intra prediction and Inter prediction, and other information such as parameters associated with loop filters applied to underlying image area.
  • the side information associated with Intra Prediction 110, Inter prediction 112 and in-loop filter 130, are provided to Entropy Encoder 122 as shown in Fig. 1A. When an Inter-prediction mode is used, a reference picture or pictures have to be reconstructed at the encoder end as well.
  • the transformed and quantized residues are processed by Inverse Quantization (IQ) 124 and Inverse Transformation (IT) 126 to recover the residues.
  • the residues are then added back to prediction data 136 at Reconstruction (REC) 128 to reconstruct video data.
  • the reconstructed video data may be stored in Reference Picture Buffer 134 and used for prediction of other frames.
  • incoming video data undergoes a series of processing in the encoding system.
  • the reconstructed video data from REC 128 may be subject to various impairments due to a series of processing.
  • in-loop filter 130 is often applied to the reconstructed video data before the reconstructed video data are stored in the Reference Picture Buffer 134 in order to improve video quality.
  • deblocking filter (DF) may be used.
  • SAO Sample Adaptive Offset
  • ALF Adaptive Loop Filter
  • the loop filter information may need to be incorporated in the bitstream so that a decoder can properly recover the required information. Therefore, loop filter information is also provided to Entropy Encoder 122 for incorporation into the bitstream.
  • DF deblocking filter
  • SAO Sample Adaptive Offset
  • ALF Adaptive Loop Filter
  • Loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in the reference picture buffer 134.
  • the system in Fig. 1A is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to the High Efficiency Video Coding (HEVC) system, VP8, VP9, H. 264 or VVC.
  • HEVC High Efficiency Video Coding
  • the decoder can use similar or portion of the same functional blocks as the encoder except for Transform 118 and Quantization 120 since the decoder only needs Inverse Quantization 124 and Inverse Transform 126.
  • the decoder uses an Entropy Decoder 140 to decode the video bitstream into quantized transform coefficients and needed coding information (e.g. ILPF information, Intra prediction information and Inter prediction information) .
  • the Intra prediction 150 at the decoder side does not need to perform the mode search. Instead, the decoder only needs to generate Intra prediction according to Intra prediction information received from the Entropy Decoder 140.
  • the decoder only needs to perform motion compensation (MC 152) according to Inter prediction information received from the Entropy Decoder 140 without the need for motion estimation.
  • Motion Compensation one of the key technologies in hybrid video coding, explores the pixel correlation between adjacent pictures. It is generally assumed that, in a video sequence, the patterns corresponding to objects or background in a frame are displaced to form corresponding objects in the subsequent frame or correlated with other patterns within the current frame. With the estimation of such displacement (e.g. using block matching techniques) , the pattern can be mostly reproduced without the need to re-code the pattern. Similarly, block matching and copy has also been tried to allow selecting the reference block from the same picture as the current block. It was observed to be inefficient when applying this concept to camera captured videos. Part of the reasons is that the textual pattern in a spatial neighbouring area may be similar to the current coding block, but usually with some gradual changes over the space. It is difficult for a block to find an exact match within the same picture in a video captured by a camera. Accordingly, the improvement in coding performance is limited.
  • a new prediction mode i.e., the intra block copy (IBC) mode or called current picture referencing (CPR)
  • IBC intra block copy
  • CPR current picture referencing
  • a prediction unit PU
  • a displacement vector called block vector or BV
  • the prediction errors are then coded using transformation, quantization and entropy coding.
  • IBC compensation is illustrated in Fig. 2, where block 212 is a corresponding block for block 210, and block 222 is a corresponding block for block 220.
  • the reference samples correspond to the reconstructed samples of the current decoded picture prior to in-loop filter operations, both deblocking and sample adaptive offset (SAO) filters in HEVC.
  • SAO sample adaptive offset
  • JCTVC-M0350 The very first version of IBC was proposed in JCTVC-M0350 (Budagavi et al., AHG8: Video coding using Intra motion compensation, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC 1/SC 29/WG11, 13th Meeting: Incheon, KR, 18–26 Apr. 2013, Document: JCTVC-M0350) to the HEVC Range Extensions (RExt) development.
  • the IBC compensation was limited to be within a small local area, with only 1-D block vector and only for block size of 2Nx2N.
  • HEVC SCC Stcreen Content Coding
  • Intra template matching prediction is a special intra prediction mode that copies the best prediction block from the reconstructed part of the current frame, whose L-shaped template matches the current template. For a predefined search range, the encoder searches for the most similar template matched with the current template in a reconstructed part of the current frame and uses the corresponding block as a prediction block. The encoder then signals the usage of this mode, and the same prediction operation is performed at the decoder side.
  • the prediction signal is generated by matching the L-shaped causal neighbour of the current block with another block in a predefined search area in Fig. 2 consisting of: R1: current CTU R2: top-left CTU R3: above CTU R4: left CTU
  • the current block 310 in R1 is matched with the corresponding block 312 in R2.
  • the templates for the current block and the matched block are shown as darker-colour L-shaped areas.
  • Area 322 corresponds to reconstructed region in the current picture 320.
  • Sum of absolute differences (SAD) is used as a cost function.
  • the decoder searches for the template that has least SAD with respect to the current one and uses its corresponding block as a prediction block.
  • ‘a’ is a constant that controls the gain/complexity trade-off. In practice, ‘a’ is equal to 5.
  • the search range of all search regions is subsampled by a factor of 2. This leads to a reduction of template matching search by 4.
  • a refinement process is performed. The refinement is done via a second template matching search around the best match with a reduced range.
  • the reduced range is defined as min (BlkW, BlkH) /2.
  • the Intra template matching tool is enabled for CUs with size less than or equal to 64 in width and height. This maximum CU size for Intra template matching is configurable.
  • the Intra template matching prediction mode is signalled at CU level through a dedicated flag when DIMD (Decoder-side Intra Mode Derivation) is not used for current CU.
  • DIMD Decoder-side Intra Mode Derivation
  • block vector (BV) derived from the intra template matching prediction (IntraTMP) is used for intra block copy (IBC) .
  • IntraTMP BV of the neighbouring blocks along with IBC BV are used as spatial BV candidates in IBC candidate list construction.
  • IntraTMP block vector is stored in the IBC block vector buffer and, the current IBC block can use both IBC BV and IntraTMP BV of neighbouring blocks as BV candidates for IBC BV candidate list as shown in Fig. 3.
  • block 410 corresponds to the current block and block 312 corresponds to a neighbouring IntraTMP block.
  • the IntraTMP BV 416 is used to locate the best matching block 422 according to the matching cost between template 424 and template 414.
  • Area 422 corresponds to reconstructed region in the current picture 430.
  • IntraTMP block vectors are added to IBC block vector candidate list as spatial candidates.
  • JVET-AC0193 (Na Zhang, et al., “Non-EE2: Copy-Padding for IBC” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 29th Meeting, by teleconference, 11–20 January 2023, Document: JVET-AC0193) , copy-padding for IBC is proposed.
  • IBC mode the reference block cannot be overlapped with the current block.
  • copy-padding can be applied to the overlapped area. With copy-padding, an unreconstructed sample in the overlapped region can be padded by copying its prediction sample as shown in Figs. 5A-B.
  • FIG. 5A illustrates an example where the current block 520 is overlapped with a reference block 510.
  • the overlapped area is shown as a dot-filled area.
  • the samples in the overlapped area are not reconstructed yet and are considered as unavailable.
  • the samples in the overlapped area can be padded by copying the prediction samples is corresponding location as shown in Fig. 5B, where samples a, b, c, e, f and g in the overlapped area of the reference block are padded by copied from corresponding predicted samples of the current block.
  • JVET-AB0191 (Changyue Ma, et al., “Non-EE2: Combining intra block copy and intra mode” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 28th Meeting, Mainz, De, 21–28 October 2022, Document: JVET-AB0191) , combination of IBC prediction with intra prediction is proposed.
  • JVET-AB0191 proposes to combine the IBC with the intra prediction.
  • the design of the combined IBC and intra prediction mode is similar to that of the existing CIIP, except that the inter part in CIIP is replaced with the IBC. Specifically, the IBC regular merge mode or IBC TM merge mode is combined with the PDPC mode or the TIMD mode.
  • a flag is first signalled to indicate whether a prediction unit is coded with combined IBC and intra prediction mode. If this flag is true, another flag is signalled to indicate whether the intra part is generated based on the TIMD or the PDPC.
  • JVET-AC0170 Japanese Patent Application Laidification (Yang Wang, et al., “Non-EE2: Fuse intra template matching prediction with intra prediction” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 29th Meeting, by teleconference, 11–20 January 2023, Document: JVET-AC0170
  • the prediction block is the weighted sum of two prediction signals generated by Intra TMP and intra prediction, in which the intra prediction mode derived by TIMD is used for intra prediction.
  • a CU flag is signalled whether the proposed method is used.
  • JVET-J0014 Description of SDR, HDR, and 360° Video Coding Technology Proposal by Fraunhofer HHI
  • JVET-J0014 M. Albrecht, et al., “Description of SDR, HDR, and 360° video coding technology proposal by Fraunhofer HHI” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 10th Meeting: San Diego, US, 10–20 Apr. 2018, Document: JVET-J0014) , some boundary padding technique are disclosed. Based on the coded block shape, the given motion vector and the number of interpolation filter taps, a particular area of the reference frame is used for motion compensated prediction.
  • Multi-Directional Boundary Padding uses angular intra prediction to extend the reference frame border, whenever the referenced pixel area is partially or entirely outside the area of the reconstructed reference frame.
  • the best fitting mode is estimated at both the encoder and the decoder side.
  • a template area is defined, which lays inside the reconstructed reference frame as shown in Fig. 6.
  • the prediction direction is rotated by 180° to point over the available border pixels inside the reference frame.
  • the template area is then predicted from the adjacent border pixels and is compared with the reconstructed reference frame pixels based on the SAD measure.
  • the angular prediction mode with the smallest template-based SAD measure is chosen, to predict the referenced pixel area outside of the reference frame.
  • the dot-filled area 610 represents an inside area of the reference frame.
  • Line 612 represents the boundary of the reference frame and circles 614 represent boundary samples of the reference frame.
  • Block 620 corresponds to a template area and block 630 corresponds to a referenced pixel area.
  • the set or arrows 640 correspond to the angular direction associated with the angular prediction mode.
  • the border pixels are only available at a single side of the predicted area. Therefore, only half of the angular intra prediction modes, e.g. either horizontal or vertical modes, are used depending on the prediction direction.
  • the angular intra prediction modes have to be rotated by 180° before applying to MDBP border extension.
  • the first template area 720 is determined, based on the outermost pixel line 742 parallel to the reference frame border.
  • the second template area 722 is determined, based on the first pixel line 740 outside the reference frame border as shown in Fig. 7, where block 730 corresponds to reference pixel area.
  • Dot-filled area 710 represents an inside area of the reference frame.
  • Line 712 represents the boundary of the reference frame and circles 714 represent boundary samples of the reference frame.
  • the referenced pixel area overlaps the frame border at two sides.
  • MDBP is only applied at one side (the side, which overlaps the frame border by most pixels) .
  • the remaining side is padded with the perpendicular frame border padding already available.
  • JVET-K0195 CE4-5 Related: Inter/Intra Boundary Padding
  • JVET-K0195 J. Brandenburg, et al., “CE4-5 related: Inter/Intra Boundary Padding” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 11th Meeting: Ljubljana, SI, 10–18 July 2018, Document: JVET-K0195
  • JVET-K0195 Joint Video Experts Team
  • reference pictures are extended by a perpendicular padding of the picture boundary samples.
  • Inter-prediction based boundary padding uses motion compensated prediction to extend the area of the reference picture.
  • the boundary extension area is divided into blocks of 4xM or Mx4 samples. Each block is filled by motion compensation using the motion information of the adjacent reference picture block. For boundary extension blocks without associated motion information and for boundary extension areas for which motion information points outside of the reference picture, fallback perpendicular padding is carried out.
  • JVET-K0363 (Yan Zhang, et al., “CE4.5.2: Motion compensated boundary pixel padding” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 11th Meeting: Ljubljana, SI, 10–18 July 2018, Document: JVET-K0363) entails addition of an average residual offset to the boundary extension samples, while the padding method in JVET-K0117 (Minsoo Park, et al., “CE4: Results on Reference picture boundary padding in J0025” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 11th Meeting: Ljubljana, SI, 10–18 July 2018, Document: JVET-K0117) supports bi-prediction of boundary extension samples.
  • Intra-prediction based boundary padding as proposed in JVET-J0012 (Rickard et al., “Description of SDR and HDR video coding technology proposal by Ericsson and Nokia” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 10th Meeting: San Diego, CA, USA, 10–20 April 2018, Document: JVET-J0012) uses angular intra-prediction to fill the area of a referenced block outside the reference picture.
  • the applied angular intra-prediction mode is chosen in encoder and decoder using a probing approach of decoded picture samples.
  • JVET-K0195 presents a harmonized boundary padding approach using inter-prediction and intra-prediction based boundary padding and reports experimental results.
  • JVET-K0195 proposes an inter/intra-prediction based boundary padding, that combines per-picture inter-prediction based boundary padding with per-reference intra-prediction based boundary padding.
  • the number of boundary padding samples originating from perpendicular boundary padding is evaluated. If this number exceeds a threshold (50%of boundary padding samples) , intra-prediction based boundary padding is used for the reference block instead.
  • JVET-K0117 CE4: Results on Reference Picture Boundary Padding in JVET-J0025
  • JVET-K0117 proposes a padding method for padding outside areas of a picture with motion compensation with motion information of edge pixel of the picture as shown in Fig. 8, where block 812 is a current block in current frame 810, and block 822 is a reference block in reference frame 820.
  • each 4x4 block is checked at the boundary of the picture. If there is motion information in the block, the location of the block is checked in the reference picture of the block. If the location is located in the image area, check whether the neighbouring area of the reference area is available.
  • the location of the neighbouring area may be located in four directions, up, down, left, and right.
  • the orientation of the adjacent area is the same as the location of the padding area. For example, if the padding area is located on the left side of the picture, then the inspection area is also on the left side of the pixel.
  • the length of the side that does not face the picture of the padding area is determined by the distance between the position of the pixel of the reference picture and the position of the edge pixel or by the size of the padding area. The shorter of them is decided. If the predetermined length is shorter than the size of the padding area, the rest of the area is filled with extrapolated edge pixels of the external picture.
  • the available adjacent area is derived by motion compensation.
  • a conventional padding method is performed when an adjacent area is unavailable or there is no information about a motion in a boundary block.
  • the block can have two pieces of information about movement. In this case, each information is used to create a padding image and integrate two images into one.
  • the last pixel of each position is extrapolated to induce a left upper portion, a right upper portion, a left lower portion, and a right padding area.
  • JVET-K0363 CE4.5.2: Motion Compensated Boundary Pixel Padding
  • the motion vector point to a reference block that is partially or entirely located outside the reference slice. Without boundary padding, these pixels will be unavailable.
  • the reference slice is padded using repetitive padding method which repeats the outer most pixel in each of the four directions for a certain amount of times depending on the padding size. These padded pixels can only provide very limited information since it is very likely that the padded area does not contain any meaningful content compare to those that lie inside the boundary.
  • JVET-K0363 a new boundary pixel padding method is introduced so that more information can be provided by the padded areas in the reference slice.
  • a motion vector is first derived from the boundary 4x4 block inside the current frame. If the boundary 4x4 block is intra coded or the motion vector is not available, repetitive padding will be used. If the boundary 4x4 block is predicted using uni-directional inter prediction. The only motion vector within the block will be used for motion compensated boundary pixel padding. Using the position of the boundary 4x4 block and its motion vector, a corresponding starting position can be computed in the reference frame.
  • a 4xM or Mx4 image slip can be fetched where M is the distance between the horizontal/vertical coordinate of the boundary pixel position and the starting position depending on the padding direction.
  • M is forced to be smaller than 64.
  • the motion vector which points to the pixel position farther away from the frame boundary in the reference slice in terms of the padding direction, is used in motion compensated boundary pixel padding.
  • the difference between the DC values of the boundary 4x4 block in the current slice and its corresponding reference 4x4 block in the reference slice is used as the offset to filter the fetched motion compensated image slip before it is copied to the padding area beyond the image boundary.
  • JVET-AA0096 EE2-2.2 Motion Compensated Picture Boundary Padding
  • bi-prediction is performed in a way that avoids relying on reference samples out of a reference picture bounds (OOB) , if possible.
  • OOB reference picture bounds
  • the OOB prediction samples are not used.
  • the concerned part of the block is rather uni-predicted based on non-OOB prediction samples, if available in the other reference picture.
  • pictures are extended by an area surrounding the picture with a size of (maxCUwidth + 16) in each direction of the picture boundary.
  • the pixel in the extended area is derived by repetitive boundary padding.
  • a reference block used for uni-prediction locates partially or completely out of the picture boundary (OOB)
  • the repetitive padded pixel is used for motion compensation (MC) .
  • JVET-Z0130 Zhi Zhang, et al., “EE2-related: Motion compensation boundary padding” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 26th Meeting, by teleconference, 20–29 April 2022, Document: JVET-Z0130
  • JVET-Z0130 a method called motion compensated boundary padding replaces the repetitive boundary padding, for increased coding efficiency.
  • JVET-AA0096 Fabrice Le Léannec, et al., “EE2-2.2: Motion compensated picture boundary padding” , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 27th Meeting, by teleconference, 13–22 July 2022, Document: JVET-AA0096
  • samples outside of the picture boundary are derived by motion compensation instead of using only repetitive padding as in ECM.
  • the total padded area size is increased by 64 (test 2.2a) or 16 (test 2.2b) compared to ECM. This is to keep MV clipping, which implements repetitive padding, non-normative.
  • Fig. 9 shows a comparison between the padding method of ECM 4.0 and the MC boundary padding method.
  • MV of a 4 ⁇ 4 boundary block is utilized to derive a M ⁇ 4 or 4 ⁇ M padding block.
  • the value M is derived as the distance of the reference block to the picture boundary as shown on Fig. 10. Moreover, M is set at least equal to 4 as soon as the motion vector points to a position internal to the reference picture bounds. If boundary block is intra coded, then MV is not available, and M is set equal to 0. If M is less than 64, the rest of the padded area is filled with the repetitive padded samples.
  • the pixels in MC padding block are corrected with an offset, which is equal to the difference between the DC values of the reconstructed boundary block and its corresponding reference block.
  • an inter CU due to the reference samples padding of the reference picture, it is possible for an inter CU to have a reference block located outside the reference picture partially or totally as illustrated in Fig. 11.
  • bi-directional motion compensation is performed to generate the inter prediction block of the current block.
  • list 0 reference block is partially out-of-boundary (OOB) while list 1 reference block is fully inside the reference picture.
  • OOB part of the motion compensated blocks usually provides less prediction efficiency because the OOB part is simply repetitive samples derived from the boundary samples within the reference picture.
  • the motion compensation (MC) predictors generated using the OOB reference samples are less effective.
  • the OOB predictors are discarded and only the non-OOB predictors are used to generate the final predictor.
  • the positions of the predictors within the current block are denoted as Pos_x i, j and Pos_y i, j
  • the MV of the current block are denoted as and (x could be 0 or 1 for list 0 and list1, respectively) .
  • Pos LeftBdry , Pos RightBdry , Pos TopBdry , Pos BottomBdry are the positions of four boundaries of a picture. Since 1/16-pel MV is used in the ECM, all variables are denoted in the unit of the 1/16 sample and thus the value of half_pixel is set equal to 8.
  • the predictor is regarded as OOB when at least one of the following conditions holds
  • the proposed scheme is applied to the bi-directional MC blocks to generate the final predictors where the following procedures are deployed.
  • this procedure is termed as OOB calibration.
  • the same checking mechanism is also applied when BCW is enabled.
  • the direct block vector is used for chroma block in dual tree slices.
  • a flag is signalled to indicate whether a chroma block 1310 is coded using IBC mode. If one of the luma blocks in five locations (labelled as TL, TR, C, BL and BR) of the current luma block 1320 as shown in Fig. 13 is coded with IBC or intraTMP mode, its block vector is scaled and is used as block vector for the chroma block.
  • template matching is used to perform block vector scaling.
  • a method and apparatus for video coding using coding tools including one or more cross component models related modes are disclosed.
  • input data associated with a current block are received, wherein the input data comprise pixel data to be encoded at an encoder side or data associated with the current block to be decoded at a decoder side.
  • a reconstructed block for the current block is generated, wherein a first portion of the reconstructed block is located within a picture boundary and a second portion of the reconstructed block is located outside the picture boundary.
  • In-loop filtering is applied to the first portion and the second portion of the reconstructed block.
  • the second portion of the reconstructed block comprises padding samples.
  • the padding samples are generated according to MV (Motion Vector) -based picture boundary padding sample generation, BV (Block Vector) -based picture boundary padding sample generation, or intra-prediction based picture boundary padding samples generation.
  • MV Motion Vector
  • BV Block Vector
  • a collocated luma block referenced is determined by said one or more chroma blocks, wherein one or more collocated luma positions are outside of the collocated luma block.
  • a scaled chroma block vector is derived for said one or more chroma blocks based on a luma block vector associated with the a luma blocks at one of said one or more collocated luma positions.
  • Said one or more chroma blocks are encoded or decoded using one or more chroma reference blocks according to the scaled chroma block vector.
  • the luma block and said one or more chroma blocks are formed using a single tree. In one embodiment, the luma block and said one or more chroma blocks are formed using a dual tree. In one embodiment, said one or more collocated luma positions are at non-adjacent spatial positions of the collocated luma block. In one embodiment, a distance between each of said one or more collocated luma positions and the collocated luma block is designed according to CU width or CU height. In one embodiment, a TM (Template Matching) refinement is applied to further refine the luma block vector associated with each of said one or more collocated luma positions.
  • TM Tempolate Matching
  • input data associated with a current block are received, wherein the input data comprise pixel data to be encoded at an encoder side or data associated with the current block to be decoded at a decoder side, and wherein the current block is coded in IBC (Intra Block Copy) mode or intraTMP (Intra Template Matching Prediction) mode.
  • IBC Intra Block Copy
  • TMP Intra Template Matching Prediction
  • a reference block is determined according to a current BV (Block Vector) of the current block. If one or more neighbouring samples of a neighbouring area for the current block are outside a picture boundary, one or more padding samples are generated for said one or more neighbouring samples by using one or more target reference samples at corresponding neighbouring locations of the reference block.
  • a current picture is encoded or decoded using information comprising said one or more padding samples.
  • the current block corresponds to a boundary block. In one embodiment, if said one or more neighbouring samples of the neighbouring area are above or on a left side of the current block, said one or more target reference samples of the reference block are above or on a left side of the reference block respectively.
  • Fig. 1A illustrates an exemplary adaptive Inter/Intra video coding system incorporating loop processing.
  • Fig. 1B illustrates a corresponding decoder for the encoder in Fig. 1A.
  • Fig. 2 illustrates an example of IBC (Intra Block Copy) compensation, where blocks are predicted by corresponding blocks in the same picture.
  • IBC Intra Block Copy
  • Fig. 3 illustrates an example of search area used for Intra template matching.
  • Fig. 4 illustrates an example of use of IntraTMP block vector for IBC block.
  • Fig. 5A illustrates an example where the current block is overlapped with a reference block, where the samples in the overlapped area are not reconstructed yet and are considered as unavailable in the conventional approach.
  • Fig. 5B illustrates an example according to JVET-AC0193, where the samples in the overlapped area can be padded by copying the prediction samples in corresponding locations.
  • Fig. 6 illustrates an example of template area used for estimating the Multi-Directional Boundary Padding (MDBP) angular mode.
  • MDBP Multi-Directional Boundary Padding
  • Fig. 7 illustrates an example of two MDBP template areas T 1 and T 2 based on the outermost and the first pixel line respectively, which lay outside of the reference frame.
  • Fig. 8 illustrates an example of boundary pixel padding using motion compensation.
  • Fig. 9 shows a comparison between the padding method of ECM 4.0 and the MC boundary padding method.
  • Fig. 10 illustrates an example of deriving a M ⁇ 4 padding block with a left padding direction.
  • Fig. 11 illustrates an example of bi-directional prediction with the out-of-boundary (OOB) process according to ECM 4.0.
  • Fig. 12 illustrates an example of bi-directional motion-compensated prediction with the out-of-boundary (OOB) process.
  • Fig. 13 illustrates an example of luma blocks used to derive direct block vector for the chroma component.
  • Fig. 14 illustrates an example of bi-predictive IBC with OOB, where samples of the first reference block covered by block lines are blended with the corresponding samples of the second reference block to generate the final samples of final predictor.
  • Fig. 15 illustrates an example of DBV for the case of 4: 2: 0 colour format according to an embodiment of the present invention.
  • Fig. 16 illustrates an example of block vectors (BVs) based picture boundary padding samples generation according to one embodiment of the present invention.
  • Fig. 17A illustrates an example of BVs of RR-IBC-coded blocks and RR-IntraTMP-coded blocks that are utilized to generate picture boundary padding samples according to one embodiment of the present invention.
  • Fig. 17B illustrates an example of in-loop filtering BEING performed at padding samples according to one embodiment of the present invention.
  • Fig. 18 illustrates an example of OOB reference block in BV-based picture boundary padding samples generation.
  • Fig. 19 illustrates an example of copy-padding applied to the overlapped area, where if the overlapped area is too large, an undesirable copy behaviour may occur, such as iterative sample copy.
  • Fig. 20 illustrates an example of iterative sample copy due to the non-overlapped samples being much less than overlapped samples.
  • the copy-padding needs to perform 7 times to fill all overlapped samples with available prediction samples.
  • Fig. 21 illustrates an example of block-based copy-padding.
  • the unavailable samples number are 4 times of CuHeight.
  • Fig. 22 illustrates a flowchart of an exemplary video coding system that applies in-loop filter to picture boundary padding samples according to an embodiment of the present invention.
  • Fig. 23 illustrates a flowchart of an exemplary video coding system, where the collocated luma position can be the positions outside the collocated luma block according to an embodiment of the present invention.
  • Fig. 24 illustrates a flowchart of an exemplary video coding system that generates picture boundary padding samples based on block vectors according to an embodiment of the present invention.
  • a reference block pointed by a BV is partially out of a valid region
  • the outside part of the reference block can be filled by using a padding technique and this reference block can be referenced by IBC mode.
  • the reference block can be filled by a padding technique and referenced by IBC mode.
  • the valid region can be a picture boundary area, a subpicture boundary area, a slice/tile boundary area or IBC valid region (i.e., current CTU row and 256 rows on above) .
  • padding techniques can be used to any screen content coding tools, such as IBC and intraTMP.
  • the final predictor is generated by blending the two reference blocks.
  • one reference block is within the valid region, and the other reference block includes at least one of sample out of the valid region
  • some samples of final predictors will be generated by blending two samples of two reference blocks
  • other samples of final predictors will be generated by referring one of reference blocks. For example, in Fig. 14, samples of the first reference block covered by block lines will be blended with the corresponding samples of the second reference block to generate the final samples of final predictor. For the remaining samples of the final predictor, they will directly reference the samples in the second reference block.
  • this bi-predictive IBC mode will fallback to uni-prediction IBC prediction. That is, only the reference block within a valid region can be used.
  • one reference block is within the valid region, and the other reference block includes at least one sample out of the valid region, some samples of final predictors will be generated by blending two samples of two reference blocks, and other samples of final predictors will be generated by referring one of reference block.
  • the behaviour of each sample in one 4x4 region shall be the same. That is, if at least one sample in one 4x4 region is out of the valid region, all samples in this 4x4 will use uni-prediction by referencing the sample from the reference block inside valid region.
  • this candidate can be used for bi-predictive IBC mode.
  • all above mentioned embodiments can be used by any screen content coding tools, such as IBC and intraTMP.
  • two BVs of a bi-predictive IBC will be stored in a motion buffer and can be referenced by other blocks.
  • the bi-predictive IBC motion can be referenced as a spatial candidate, HMVP candidate, or NAMVP candidate.
  • the bi-predictive IBC motion can be referenced as an IBC predictor for IBC-GPM. That is, the bi-predictive reference block will be further combined with an intra predictor or a uni-prediction predictor.
  • a first merge list can be generated by using all bi-predictive IBC motions. After that, the bi-predictive IBC motions will be further separated to two individual uni-prediction motions to generate a second merge list.
  • the second merge list can be used for IBC-GPM.
  • a first merge list can be generated by using all bi-predictive IBC motions. After that, the bi-predictive IBC motions will be further separated into two individual uni-prediction motions to generate a second merge list. After that, those uni-prediction motion can be used to generate more bi-predictive IBC candidates.
  • BCW can be used in bi-predictive IBC. That is, two reference blocks of bi-predictive IBC can be blended with non-equal weights.
  • bi-prediction LIC Longel Illuminance Compensation
  • RR-IBC reconstruction-Reordered IBC
  • the RRIBC type or two reference blocks shall be the same.
  • both reference blocks use horizontal flipping.
  • both reference blocks use vertical flipping.
  • the collocated luma position can be the positions outside the collocated luma block.
  • the collocated luma position can be a top-left spatial neighbouring position, a bottom-left spatial neighbouring position, a top-right spatial neighbouring position, or other non-adjacent spatial positions.
  • Fig. 15 illustrates an example of DBV for the case of 420 colour format according to an embodiment of the present invention.
  • the luma signal 1510 with current luma block 1520 is shown on the left and the chroma signal 1530 with current chroma block 1540 is shown on the right.
  • Location 1522 indicates the location of the collocated current luma block 1520 and location 1542 indicates the location of the current chroma block 1540. Based on the conventional approach, only location 1522 is allowed to be the collocated luma position. According to an embodiment of the present invention, the positions (indicated by triangles in 1510) outside the collocated luma block can be used to derive prediction for the chroma block. In this embodiment, the collocated luma positions are top-left spatial neighbouring positions. As mentioned above, the collocated luma positions can also be bottom-left spatial neighbouring positions, top-right spatial neighbouring positions, or other non-adjacent spatial positions.
  • the collocated luma position can be the positions outside the collocated luma block.
  • the positions are non-adjacent spatial positions of collocated luma block.
  • the distance between the reference positions and collocated luma block is designed according to CU width or CU height.
  • the referenced BV for chroma block is derived from the collocated luma block.
  • the referenced BV for chroma block is derived from the collocated luma block.
  • TM refinement is applied to further refine the reference BV from the collocated luma block.
  • MVs of inter-coded boundary blocks are utilized to generate picture boundary padding samples for reference picture.
  • BVs of IBC-coded blocks and IntraTMP-coded blocks are exploited to generate picture boundary padding samples according to the present invention.
  • Picture boundary padding samples may or may not consider reconstruction-reordering in IBC mode and IntraTMP mode.
  • In-loop filtering may or may not be applied to picture boundary padding samples to reduce artefact between samples inside picture boundary and padding samples outside picture boundary.
  • Out-of-boundary check may be applied during padding samples generation. Chroma padding samples may be considered in the proposed method.
  • BVs of IBC-coded blocks and IntraTMP-coded blocks are utilized to generate picture boundary padding samples, as shown in Fig. 16.
  • BV of boundary block is used and the samples above reference block are used as padding samples in padding area.
  • the current picture 1610 is padded to include the padding area 1620.
  • Block 1630 corresponds to an IBC coded block at the boundary.
  • the picture boundary padding samples 1632 can be derived using BV 1650 to locate the reference block 1640.
  • the corresponding area 1642 is used as the padding samples for padding area 1632.
  • BVs of boundary blocks are used to generate picture boundary padding samples.
  • only horizontal BV component of boundary blocks is used to generate picture boundary padding samples.
  • only vertical BV component of boundary blocks is used to generate picture boundary padding samples.
  • BV component utilized in picture boundary padding samples generation may depend on padding sample locations. For example, when generating top picture boundary padding samples, only vertical BV component is considered. In another example, when generating right picture boundary padding samples, only horizontal BV component is considered.
  • boundary block is not IBC-coded or IntraTMP-coded
  • MV or intra-prediction-based method or repetitive padding is performed to derive padding samples.
  • neighbouring boundary block’s BV is used to derive padding samples for current boundary padding samples.
  • BV validity check is performed to ensure neighbouring BV is valid at current block’s reference region.
  • the validity check can be but not limited to, CTU reference region, current block reference region, etc.
  • BVs of RR-IBC-coded blocks and RR-IntraTMP-coded blocks can be utilized to generate picture boundary padding samples, as shown in Fig. 17A.
  • BV of boundary block is used and the samples above reference block (Region B 1742) or below reference block (Region A 1744) are used as padding samples in padding area.
  • the current picture 1710 is padded to include the padding area 1720.
  • Block 1730 corresponds to an RR-IBC or RR-IntraTMP coded block at the boundary.
  • the picture boundary padding samples 1732 can be derived using BV 1750 to locate the reference block 1740.
  • region A is used as picture boundary padding samples.
  • region B is used as picture boundary padding samples.
  • region A and region B are both used as picture boundary padding samples.
  • repetitive padding or intra-prediction-based padding or MV-based padding method is used for picture boundary padding samples generation.
  • in-loop filtering is not applied to picture boundary padding samples.
  • artefacts might exist between in-loop filtered samples inside picture and un-filtered samples outside picture.
  • in-loop filtering may be applied to picture boundary padding samples.
  • the proposed method can be applied to all different kinds of padding methods, including MV-based picture boundary padding samples generation, BV-based picture boundary padding samples generation, intra-prediction method-based picture boundary samples generation and repetitive picture boundary samples padding.
  • in-loop filtering will be applied to both reconstruction samples inside picture and padding samples outside picture when MV-based picture boundary padding samples generation is performed.
  • in-loop filtering will be applied to both reconstruction samples inside picture and padding samples outside picture when BV-based picture boundary padding samples generation is performed.
  • in-loop filtering will be applied to both reconstruction samples inside picture and padding samples outside picture when intra-prediction method-based picture boundary padding samples generation is performed.
  • the in-loop filtering type when in-loop filtering is performed at padding samples (Region B 1782 in Fig. 17B) , the in-loop filtering type is all the same or partially the same as in-looping filtering types used in samples inside picture (Region A 1780 in Fig. 17B) .
  • the in-loop filtering type is, for example but not limited to, filter index, filter set, filter coefficients, geometry transform, loop filtering input sample sources, etc.
  • the current picture 1760 is padded to include the padding area 1770.
  • the in-loop filtering type when in-loop filtering is performed at padding samples (Region B 1782in Fig. 17B) , the in-loop filtering type is all the same or partially the same as in-looping filtering types used in samples outside picture (padding area in Fig. 17B) .
  • the in-loop filtering type is, for example but not limited to, filter index, filter set, filter coefficients, geometry transform, loop filtering input sample sources, etc.
  • padding samples can partially or all from samples inside picture, or the padding samples generation will be terminated because of unavailable samples.
  • BV of boundary block is used and the samples above reference block are used as padding samples in padding area as shown in Fig. 18.
  • the current picture 1810 is padded to include the padding area 1820.
  • Block 1830 corresponds to an IBC coded block at the boundary.
  • the picture boundary padding samples 1832 can be derived using BV 1850 to locate the reference block 1840, which is partially outside the picture boundary.
  • the corresponding area 1842 are used as the padding samples for padding area 1832.
  • BV of IBC-coded or IntraTMP-coded blocks points to out-of-boundary block
  • only the non-OOB BV component is used in padding samples generation.
  • horizontal BV component points to OOB
  • BV of IBC-coded or IntraTMP-coded blocks points to out-of-boundary block
  • only corresponding BV direction considering padding direction is used. For example, when performing top picture boundary padding, only use vertical BV component in padding samples generation. When performing right picture boundary padding, only use horizontal BV component in padding samples generation.
  • BV of IBC-coded or IntraTMP-coded blocks points to out-of-boundary block
  • only the inside picture samples are used as padding samples.
  • For samples outside picture it is not used in padding samples generation and repetitive padding or intra-prediction based method is utilized.
  • padding samples generation is terminated and repetitive padding or intra-prediction-based padding method is utilized to generate padding samples.
  • chroma padding samples may be considered in IBC mode and IntraTMP mode, regardless whether chroma predictor is generated or not in IBC mode or IntraTMP mode.
  • chroma padding samples are also generated, using the BV with chroma format considered (for example, 4: 0: 0 or 4: 2: 0 or 4: 4: 4) .
  • chroma padding samples are also generated, but only using the horizontal BV with chroma format considered (for example, 4: 0: 0 or 4: 2: 0 or 4: 4: 4) .
  • chroma padding samples are also generated, but only using the vertical BV with chroma format considered (for example, 4: 0: 0 or 4: 2: 0 or 4: 4: 4) .
  • chroma padding samples are also generated, but only using the vertical BV with chroma format considered (for example, 4: 0: 0 or 4: 2: 0 or 4: 4: 4) .
  • fractional motion resolution can be applied to the intraTMP searching process in some cases. For example, if the current sequence is not screen content, both integer-pel and half-pel motion resolution are tested and the best motion position is selected for final prediction. If the current sequence is screen content, only integer-pel motion resolution can be applied.
  • whether to use fractional-pel motion resolution in the searching process can be determined by a high-level flag.
  • a flag can be signalled in Sequence Parameter Set (SPS) or Picture Parameter Set (PPS) to indicate whether fractional-pel resolution is allowed for intraTMP or not.
  • SPS Sequence Parameter Set
  • PPS Picture Parameter Set
  • a flag can be signalled in picture header or slice header, such as ph_frac_intraTMP_enabled, sh_frac_intraTMP_enabled.
  • a higher level syntax such as sps_frac_intraTMP_enabled or pps_frac_intraTMP_enabled is signalled. Only if it is true, the lower level related syntax, such as ph_frac_intraTMP_enabled, sh_frac_intraTMP_enabled, can be signalled. Additionally, if the related syntax, such as ph_frac_intraTMP_enabled, is signalled in picture header, the related syntax signalled in slice header (e.g. sh_frac_intraTMP_enabled) will not be signalled. In other words, only if the related syntax (e.g. ph_frac_intraTMP_enabled) is not present in picture header, the related syntax in slice header (e.g. sh_frac_intraTMP_enabled) can be signalled.
  • the related syntax e.g. ph_frac_intraTMP_enabled
  • slice header e.g. sh_
  • the interpolation filter used in fractional-pel resolution of intraTMP can be determined by high level syntaxes. For example, sps_frac_interpolation_filter_index or pps_frac_interpolation_filter_index is signalled. For another example, only if sps_frac_intraTMP_enabled or pps_frac_intraTMP_enabled is true, sps_frac_interpolation_filter_index or ppsfrac_interpolation_filter_index can be signalled.
  • ph_frac_interpolation_filter_index or sh_frac_interpolation_filter_index can be signalled.
  • ph_frac_interpolation_filter_index or sh_frac_interpolation_filter_index can be signalled.
  • sps_frac_interpolation_filter_index, pps_frac_interpolation_filter_index, ph_frac_interpolation_filter_index or sh_frac_interpolation_filter_index 0
  • 8-tap interpolation filter 8-tap interpolation filter
  • sps_frac_interpolation_filter_index, pps_frac_interpolation_filter_index, ph_frac_interpolation_filter_index or sh_frac_interpolation_filter_index means 4-tap interpolation filter is used.
  • sps_frac_interpolation_filter_index, pps_frac_interpolation_filter_index, ph_frac_interpolation_filter_index or sh_frac_interpolation_filter_index means bilinear interpolation filter is used.
  • ph_frac_intraTMP_enabled, or sh_frac_intraTMP_enabled is true
  • ph_frac_interpolation_filter_8 or sh_frac_interpolation_filter_8 can be signalled.
  • ph_frac_interpolation_filter_8 or sh_frac_interpolation_filter_8 equal to 1, means 8-tap interpolation filter is used.
  • ph_frac_interpolation_filter_8 or shfrac_interpolation_filter_8 equal to 0, means 4-tap interpolation filter is used.
  • fusion can be applied to intraTMP. For example, if the current sequence is not screen content, during the searching process, two prediction blocks corresponding to the best and second-best TM cost are found. The final predictor can be derived by fusion of the two prediction blocks. If the current sequence is screen content, fusion technology of intraTMP will be disabled.
  • whether to use fusion technique in intraTMP can be determined by a high-level flag.
  • a flag can be signalled in SPS or PPS to indicate whether fusion technology is allowed for intraTMP.
  • a flag can be signalled in picture header or slice header, such as ph_fusion_intraTMP_enabled or sh_fusion_intraTMP_enabled.
  • a higher level syntax such as sps_fusion_intraTMP_enabled or pps_fusion_intraTMP_enabled is signalled. Only if it is true, the lower level related syntax, such as ph_fusion_intraTMP_enabled or sh_fusion_intraTMP_enabled, can be signalled. Additionally, if the related syntax, such as ph_fusion_intraTMP_enabled, is signalled in picture header, the related syntax signalled in slice header (e.g. sh_fusion_intraTMP_enabled) won't be signalled. In other words, only if the related syntax (e.g. ph_fusion_intraTMP_enabled) is not present in picture header, the related syntax in slice header (e.g. sh_fusion_intraTMP_enabled) can be signalled.
  • the related syntax e.g. ph_fusion_intraTMP_enabled
  • slice header e.g. sh
  • a filter is applied to adapt the characteristics of the copied block to the local neighbourhood.
  • a higher level syntax is used to control the on-off of filtered template matching based intra prediction, such as sps_fusion_intraTMP_enabled or pps_fusion_intraTMP_enabled.
  • a flag can be signalled in picture header or slice header, such as ph_filtered_intraTMP_enabled or sh_filtered_intraTMP_enabled.
  • a higher level syntax such as sps_filtered_intraTMP_enabled or pps_filtered_intraTMP_enabled is signalled.
  • the lower level related syntax such as ph_filtered_intraTMP_enabled or sh_filtered_intraTMP_enabled
  • the related syntax such as ph_filtered_intraTMP_enabled
  • the related syntax signalled in slice header e.g. sh_filtered_intraTMP_enabled
  • the related syntax in slice header e.g. sh_filtered_intraTMP_enabled
  • IBC intra block copy mode
  • fractional motion resolution, fusion technology or filtered prediction can be applied to IBC.
  • the on-off can be determined by a high-level flag, such as SPS flags, PPS flags, PH flags, or SH flags.
  • a fractional motion TM refinement is applied to a BVP after IBC merge list or AMVP list generation, where the precision can be half-pel, or quarter-pel.
  • the TM cost of fractional refined BVPs can be used to reorder IBC merge list or IBC AMVP list.
  • N can be any integer larger than 0.
  • the integer TM costs can be stored during TM refinement stage.
  • parametric error surface equation can be used to further refine BVPs in fractional motion precision. For example, quarter-pel can be applied.
  • fractional TM refinement or parametric error surface equation is used to refine IBC BVPs.
  • the selection is determined by integer TM costs.
  • fractional TM refinement and parametric error surface equation are sequentially applied.
  • a higher-level syntax is used to control the on-off of fractional BV refinement on IBC, such as sps_frac_ibc_enabled, or pps_frac_ibc_enabled.
  • a flag can be signalled in picture header or slice header, such as ph_frac_ibc_enabled, sh_frac_enabled_enabled.
  • a higher level syntax such as sps_frac_ibc_enabled or pps_frac_ibc_enabled is signalled.
  • the lower level related syntax such as ph_frac_ibc_enabled or sh_frac_ibc_enabled
  • the related syntax such as ph_frac_ibc_enabled
  • the related syntax signalled in slice header e.g. sh_frac_ibc_enabled
  • the related syntax in slice header e.g. sh_frac_ibc_enabled
  • the on-off of fractional BV refinement on IBC is determined by a hash hit rate. Only if a hash hit rate is larger than a pre-defined threshold, IBC with fractional BV refinement can be enabled.
  • a flag specifying that the merge mode with motion vector difference uses only integer sample precision for the current picture can be used to determine the on-off of fractional BV refinement.
  • the usage of fractional TM refinement or parametric error surface equation can be determined by a high-level flag.
  • copy-padding can be applied on the overlapped area.
  • an undesirable copy behaviour may occur, such as iterative sample copy, as shown in Fig. 19.
  • the unreconstructed samples (grey samples) in the reference block are padded by copying their prediction samples. Since non-overlapped samples are much less than overlapped samples, the copy-padding needs to perform 7 times to fill all overlapped samples with available prediction samples as shown in Fig. 20.
  • the copy-padding technique cannot be applied if the horizontal BV component is smaller than or equal to minus half of CuWidth or the vertical BV component is smaller than or equal to minus half of CuHeight. In another embodiment, the copy-padding technique cannot be applied if the horizontal BV component is smaller than minus half of CuWidth or the vertical BV component is smaller than minus half of CuHeight.
  • the copy-padding technology cannot be applied if the copy-padding needs more than N rounds to fill the overlapped region.
  • N can be any integer larger than 0.
  • the overlapped region mentioned above can be replaced by all kinds of unavailable regions, such as the unreconstructed region or the region not allowed for IBC to reference.
  • copy-padding technique is allowed for a reference block needed more than one round to fill the overlapped region. If a reference block is overlapped with unavailable samples, at most N rounds of copy-padding technology are applied. After N rounds of copy-padding technique are performed, if there are still unavailable samples on the reference block, intra planer method are used to fill all unavailable samples on the reference block. N can be any integer larger than 0.
  • the above-mentioned intra prediction method can be replaced by position dependent intra prediction combination (PDPC) , intra DC mode, or any other intra prediction method (e.g. TIMD or DIMD) , or directly filling the available region with a pre-defined value.
  • PDPC position dependent intra prediction combination
  • intra DC mode intra DC mode
  • any other intra prediction method e.g. TIMD or DIMD
  • directly filling the available region with a pre-defined value can be designed based on sequence’s bit depth, or signalled in the bitstream (e.g. SPS, PPS, PH or SH level) .
  • the above-mentioned constraints are only used when the unavailable region includes at least one corner of the reference block.
  • the unavailable region of a reference block are R rows of CuWidth or C columns of CuHeight, it is allowed to apply copy-padding technique more than one time until all unavailable samples are filled.
  • non-available region of Ref1 block includes 3 columns of CuHeight.
  • copy-padding technique can only be applied when the unavailable samples are N rows of CuWidth or M rows of CuHeights.
  • N and M shall be multiple of K integers.
  • a block-based copy-padding technique can be applied. For example, if K is equal to 4, only if N and M are both multiple of 4, a block-based copy-padding technique can be applied, where the availability of each 4x4 will be checked together. Once one 4x4 is unavailable, 16 samples in that 4x4 will be filled by the corresponding available prediction samples.
  • the block-based copy-padding technique is simpler than sample-based copy-padding technique since the invalid checking does not need to be performed for each sample.
  • IBC with intra mode is disclosed.
  • IBC predictor and intra predictor will be blended to generate the final IBC-CIIP predictor.
  • IBC predictor includes the non-available samples, IBC-CIIP cannot be performed. It will be disabled.
  • JVET-AC0170 fusing intra template matching prediction with intra prediction is disclosed.
  • IBC or intraTMP related blending modes can be enabled in more cases.
  • the corresponding samples of the final predictor will be filled by the second predictor without blending.
  • IBC or intraTMP predictor if some samples of IBC or intraTMP predictor are unavailable, the above mentioned methods in “Copy-Padding Technique” section can be applied to fill unavailable samples with available prediction samples. After that, IBC or intraTMP predictor can be blended with the second predictor by a pre-defined weighting pair. The weights applied to IBC or intraTMP predictor are smaller than or equal to the weights applied to the second predictor.
  • weighting pairs can be designed based on the percentage of original available sample number of a predictor, which is the number of available sample number before copy-padding technique. For example, if an 8x8 IBC predictor consists of 16 unavailable samples and 48 available samples. The weight of second predictor will be 4 times the weight of IBC predictor. For another embodiment, weighting pairs can be designed based on the percentage of original available sample number of a predictor. The sum of weight values of two predictors shall be a power of 2 value. After that, the final blending predictors can be generated by a right shift operation.
  • the above mentioned methods can also be applied to IBC with GPM.
  • any of the proposed padding methods can be implemented in an Intra/Inter coding module (e.g. Intra Pred. 150/MC 152 in Fig. 1B) in a decoder or an Intra/Inter coding module is an encoder (e.g. Intra Pred. 110/Inter Pred. 112 in Fig. 1A) .
  • Any of the proposed padding methods can also be implemented as a circuit coupled to the intra/inter coding module at the decoder or the encoder.
  • the decoder or encoder may also use additional processing unit to implement the padding processing. While the Intra Pred. units (e.g. unit 110/112 in Fig.
  • FIG. 1A and unit 150/152 in Fig. 1B are shown as individual processing units, they may correspond to executable software or firmware codes stored on a media, such as hard disk or flash memory, for a CPU (Central Processing Unit) or programmable devices (e.g. DSP (Digital Signal Processor) or FPGA (Field Programmable Gate Array) ) .
  • a CPU Central Processing Unit
  • programmable devices e.g. DSP (Digital Signal Processor) or FPGA (Field Programmable Gate Array) .
  • Fig. 22 illustrates a flowchart of an exemplary video coding system that applies in-loop filter to picture boundary padding samples according to an embodiment of the present invention.
  • the steps shown in the flowchart may be implemented as program codes executable on one or more processors (e.g., one or more CPUs) at the encoder side.
  • the steps shown in the flowchart may also be implemented based hardware such as one or more electronic devices or processors arranged to perform the steps in the flowchart.
  • input data associated with a current block are received in step 2210, wherein the input data comprise pixel data to be encoded at an encoder side or data associated with the current block to be decoded at a decoder side.
  • a reconstructed block for the current block is generated in step 2220, wherein a first portion of the reconstructed block is located within a picture boundary and a second portion of the reconstructed block is located outside the picture boundary.
  • In-loop filtering is applied to the first portion and the second portion of the reconstructed block in step 2230.
  • Fig. 23 illustrates a flowchart of an exemplary video coding system, where the collocated luma position can be the positions outside the collocated luma block according to an embodiment of the present invention.
  • input data associated with a current block are received in step 2310, wherein the input data comprise pixel data to be encoded at an encoder side or data associated with the current block to be decoded at a decoder side, wherein DBV (Direct Block Vector) mode is enabled for the current block.
  • DBV Direct Block Vector
  • a collocated luma block referenced by said one or more chroma blocks is determined in step 2320, wherein one or more collocated luma positions are outside of the collocated luma block.
  • a scaled chroma block vector is derived for said one or more chroma blocks based on a luma block vector associated with the a luma blocks at one of said one or more collocated luma positions in step 2330.
  • Said one or more chroma blocks are encoded or decoded using one or more chroma reference blocks according to the scaled chroma block vector in step 2340.
  • Fig. 24 illustrates a flowchart of an exemplary video coding system that generates picture boundary padding samples based on block vectors according to an embodiment of the present invention.
  • input data associated with a current block are received in step 2410, wherein the input data comprise pixel data to be encoded at an encoder side or data associated with the current block to be decoded at a decoder side, and wherein the current block is coded in IBC (Intra Block Copy) mode or intraTMP (Intra Template Matching Prediction) mode.
  • IBC Intra Block Copy
  • TMP Intra Template Matching Prediction
  • one or more padding samples are generated for said one or more neighbouring samples by using one or more target reference samples at corresponding neighbouring locations of the reference block in step 2430.
  • a current picture is encoded or decoded using information comprising said one or more padding samples in step 2440.
  • Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both.
  • an embodiment of the present invention can be one or more circuit circuits integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein.
  • An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein.
  • DSP Digital Signal Processor
  • the invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA) .
  • These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention.
  • the software code or firmware code may be developed in different programming languages and different formats or styles.
  • the software code may also be compiled for different target platforms.
  • different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne des procédés et un appareil de génération d'échantillon de remplissage de limite d'image. Dans un procédé, un bloc reconstruit pour le bloc courant est généré, une première partie du bloc reconstruit étant située à l'intérieur d'une limite d'image et une seconde partie du bloc reconstruit étant située à l'extérieur de la limite d'image. Un filtrage en boucle est appliqué à la première partie et à la seconde partie du bloc reconstruit. Dans un autre procédé, un bloc de luminance colocalisé référencé par un bloc de chrominance est déterminé, une ou plusieurs positions de luminance colocalisées étant à l'extérieur du bloc de luminance colocalisé. Un vecteur de bloc de chrominance mis à l'échelle est déterminé pour le bloc de chrominance sur la base d'un vecteur de bloc de luminance associé aux blocs de luminance à l'une desdites une ou plusieurs positions de luminance colocalisées. Ledit ou lesdits blocs de chrominance sont codés à l'aide d'un bloc de référence de chrominance selon le vecteur de bloc de chrominance mis à l'échelle.
PCT/CN2024/073213 2023-01-19 2024-01-19 Procédés et appareil de génération d'échantillon de limite pour une copie intra-bloc et mise en correspondance intra-modèle dans un codage vidéo Ceased WO2024153220A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP24744390.6A EP4652739A1 (fr) 2023-01-19 2024-01-19 Procédés et appareil de génération d'échantillon de limite pour une copie intra-bloc et mise en correspondance intra-modèle dans un codage vidéo

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US202363480551P 2023-01-19 2023-01-19
US63/480551 2023-01-19
US202363494498P 2023-04-06 2023-04-06
US63/494498 2023-04-06
US202363512312P 2023-07-07 2023-07-07
US63/512312 2023-07-07

Publications (2)

Publication Number Publication Date
WO2024153220A1 WO2024153220A1 (fr) 2024-07-25
WO2024153220A9 true WO2024153220A9 (fr) 2025-10-16

Family

ID=91955345

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/073213 Ceased WO2024153220A1 (fr) 2023-01-19 2024-01-19 Procédés et appareil de génération d'échantillon de limite pour une copie intra-bloc et mise en correspondance intra-modèle dans un codage vidéo

Country Status (2)

Country Link
EP (1) EP4652739A1 (fr)
WO (1) WO2024153220A1 (fr)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10728573B2 (en) * 2017-09-08 2020-07-28 Qualcomm Incorporated Motion compensated boundary pixel padding
WO2019125093A1 (fr) * 2017-12-22 2019-06-27 주식회사 윌러스표준기술연구소 Procédé et dispositif de traitement du signal vidéo
WO2021004491A1 (fr) * 2019-07-09 2021-01-14 Beijing Bytedance Network Technology Co., Ltd. Détermination d'échantillon pour filtrage à boucle adaptatif
CN117063470A (zh) * 2021-02-20 2023-11-14 抖音视界有限公司 图像/视频编解码中的边界上的填充样点滤波
WO2022174784A1 (fr) * 2021-02-20 2022-08-25 Beijing Bytedance Network Technology Co., Ltd. Écrêtage de vecteurs de mouvement par remplissage de limites dans un codage d'images/vidéo

Also Published As

Publication number Publication date
WO2024153220A1 (fr) 2024-07-25
EP4652739A1 (fr) 2025-11-26

Similar Documents

Publication Publication Date Title
US11785207B2 (en) Apparatus of encoding or decoding video blocks by current picture referencing coding
WO2023198142A1 (fr) Procédé et appareil de prédiction implicite de composantes croisées dans un système de codage vidéo
WO2023241637A9 (fr) Procédé et appareil de prédiction inter-composantes avec mélange dans des systèmes de codage vidéo
WO2023208220A1 (fr) Procédé et appareil pour réordonner des candidats de fusion avec un mode mvd dans des systèmes de codage vidéo
WO2024017061A1 (fr) Procédé et appareil de remplissage d'image dans un codage vidéo
WO2023208224A1 (fr) Procédé et appareil de réduction de complexité de codage vidéo à l'aide de fusion avec mode mvd
WO2024153198A1 (fr) Procédés et appareil de vecteurs de blocs fractionnaires dans une copie de bloc intra et une mise en correspondance de modèles intra pour un codage vidéo
WO2024153220A9 (fr) Procédés et appareil de génération d'échantillon de limite pour une copie intra-bloc et mise en correspondance intra-modèle dans un codage vidéo
WO2023020389A1 (fr) Procédé et appareil de mise en correspondance de modèles à faible latence dans un système de codage vidéo
WO2025218731A1 (fr) Procédé et appareil de remplissage de limite d'image reposant sur un modèle dans un codage vidéo
WO2024149017A1 (fr) Procédés et appareil de décalage de mouvement dans une compensation de mouvement de blocs chevauchés pour codage vidéo
WO2023208189A1 (fr) Procédé et appareil pour l'amélioration d'un codage vidéo à l'aide d'une fusion avec un mode mvd avec mise en correspondance de modèles
WO2025026397A1 (fr) Procédés et appareil de codage vidéo utilisant une prédiction inter-composantes à hypothèses multiples pour un codage de chrominance
WO2024149285A1 (fr) Procédé et appareil de prédiction intra d'appariement modèle-objet pour un codage vidéo
WO2024213104A1 (fr) Procédés et appareil de copie de bloc intra avec prédiction d'hypothèse multiple pour codage vidéo
WO2025148935A1 (fr) Procédé et appareil de mélange basé sur une régression destiné à améliorer une inter-prédiction dans un système de codage vidéo
WO2025007931A1 (fr) Procédés et appareil d'amélioration de codage vidéo par de multiples modèles
WO2024027784A1 (fr) Procédé et appareil de prédiction de vecteurs de mouvement temporel basée sur un sous-bloc avec réorganisation et affinement dans un codage vidéo
WO2025218694A1 (fr) Procédés et appareil de sélection du nombre de cadidats mvd en mode amvp avec sbtmvp pour le codage vidéo
WO2025153018A1 (fr) Procédés et appareil de candidats à bi-prédiction pour prédiction de vecteur de bloc auto-relocalisé ou prédiction de vecteur de mouvement chaîné
WO2025007972A1 (fr) Procédés et appareil visant à obtenir des modèles de composante transversale à partir de voisins temporels et historiques pour un codage inter de chrominance
WO2023143325A1 (fr) Procédé et appareil de codage vidéo utilisant un mode fusion avec mvd
WO2025153050A1 (fr) Procédés et appareil de prédiction intra basée sur un filtre avec de multiples hypothèses dans des systèmes de codage vidéo
WO2025157012A1 (fr) Procédés et appareil de fusion de signaux de prédiction intra dans une image et système de codage vidéo
WO2024169882A1 (fr) Procédés et appareil de vecteur de mouvement initial priorisé pour affinement de mouvement côté décodeur dans un codage vidéo

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24744390

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 2024744390

Country of ref document: EP