WO2025150789A1 - Intra prediction-based image coding method, and device therefor - Google Patents

Abstract

An image decoding method according to an embodiment of the present disclosure comprises the steps of: acquiring prediction-related information through a bitstream; deriving a most probable mode (MPM) list for the current block; deriving an intra prediction mode of the current block on the basis of the MPM list and the prediction-related information; and generating prediction samples for the current block on the basis of the derived intra prediction mode, wherein the MPM list includes a first MPM list, a second MPM list and a third MPM list, the first MPM list not including candidates included in the second MPM list, and the third MPM list including candidates included in the second MPM list.

Description

Intra prediction based image coding method and device thereof

The present disclosure relates to an image/video coding method and a device therefor.

Image/video coding is used in various applications such as digital storage media, television broadcasting, video streaming services, and real-time communications, and recently, the demand for high-resolution, high-quality images/videos is increasing in various fields.

As the image/video becomes higher resolution and higher quality, the data size of the image/video increases, and the amount of information or bits transmitted relatively increases. Therefore, when transmitting image data using media such as existing wired/wireless broadband lines or storing image/video data using existing storage media, the transmission cost and storage cost increase.

In addition, interest in and demand for immersive media such as VR (virtual reality), AR (artificial reality), MR (mixed reality) content and holograms have been increasing recently, and attempts to provide immersive experiences using immersive media in games, education, medicine, real estate, marketing, etc. are increasing.

Accordingly, a highly efficient image/video compression technology is required to effectively compress, transmit, store, and play back information of high-resolution, high-quality images/videos with various characteristics as described above.

According to one embodiment of the present disclosure, a method and device for improving video/image coding efficiency are provided.

According to one embodiment of the present disclosure, a method and device for intra prediction-based video/image coding are provided.

According to one embodiment of the present disclosure, a video decoding method performed by a decoding device is provided. The method includes the steps of obtaining prediction-related information through a bitstream, deriving an MPM (most probable mode) list for a current block, deriving an intra prediction mode of the current block based on the MPM list and the prediction-related information, and generating prediction samples for the current block based on the derived intra prediction mode, wherein the MPM list includes a first MPM list, a second MPM list, and a third MPM list, and candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list.

According to one embodiment of the present disclosure, a video encoding method performed by an encoding device is provided. The method includes the steps of deriving an MPM (most probable mode) list for a current block, and the step of deriving an intra prediction mode of the current block, the step of generating prediction-related information for indicating the intra prediction mode of the current block based on the MPM list, and the step of encoding video information including the prediction-related information, wherein the MPM list includes a first MPM list, a second MPM list, and a third MPM list, and candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list.

According to one embodiment of the present disclosure, a decoding device for image decoding is provided. The decoding device includes a memory and at least one processor connected to the memory, and the at least one processor is configured to perform the steps of obtaining prediction-related information through a bitstream, deriving an MPM (most probable mode) list for a current block, deriving an intra-prediction mode of the current block based on the MPM list and the prediction-related information, and generating prediction samples for the current block based on the derived intra-prediction mode, wherein the MPM list includes a first MPM list, a second MPM list, and a third MPM list, and candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list.

According to one embodiment of the present disclosure, an encoding device for video encoding is provided. The encoding device includes a memory and at least one processor connected to the memory, and the at least one processor is configured to perform the steps of deriving an MPM (most probable mode) list for a current block, deriving an intra prediction mode of the current block, generating prediction-related information for indicating the intra prediction mode of the current block based on the MPM list, and encoding video information including the prediction-related information, wherein the MPM list includes a first MPM list, a second MPM list, and a third MPM list, and candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list.

According to one embodiment of the present disclosure, a method of transmitting video/video data including a bitstream generated by a video/video encoding method according to at least one of the embodiments of the present disclosure is provided.

According to one embodiment of the present disclosure, a device is provided for transmitting video/video data including a bitstream generated by a video/video encoding method according to at least one of the embodiments of the present disclosure.

According to one embodiment of the present disclosure, a computer-readable storage medium having stored thereon a program for performing a method according to at least one of the embodiments of the present disclosure may be provided.

According to one embodiment of the present disclosure, a computer-readable digital storage medium storing encoded video/video information generated by a video/video encoding method according to at least one of the embodiments of the present disclosure is provided.

According to one embodiment of the present disclosure, a computer-readable digital storage medium is provided having encoded information or encoded video/image information stored thereon, which causes a decoding device to perform a video/image decoding method according to at least one of the embodiments of the present disclosure.

According to one embodiment of the present disclosure, the overall video/image compression efficiency can be improved.

According to one embodiment of the present disclosure, prediction performance for a current block can be improved.

According to one embodiment of the present disclosure, an efficient intra prediction mode candidate list for intra prediction mode signaling can be constructed.

Figure 1 schematically illustrates an example of a video/image coding system to which embodiments of the present disclosure can be applied.

FIG. 2 is a drawing schematically illustrating the configuration of a video/image encoding device to which embodiments of the present disclosure can be applied.

FIG. 3 is a drawing schematically illustrating the configuration of a video/image decoding device to which embodiments of the present disclosure can be applied.

Figure 4 illustrates an intra prediction procedure as an example.

Figure 5 shows examples of intra prediction based video/image encoding methods.

Figure 6 shows examples of intra prediction based video/image decoding methods.

Figure 7 shows examples of directional intra prediction modes.

Figure 8 shows an example of template-based HoG computation in DIMD.

Figure 9 shows examples of peripheral blocks for deriving the MPM list.

Figure 10 exemplarily shows DIMD-based modes and intra prediction modes derived from surrounding blocks.

Figure 11 exemplarily shows intra prediction modes of previous blocks within a certain area.

FIG. 12 schematically illustrates a video/image encoding method according to an embodiment(s) of the present disclosure.

FIG. 13 schematically illustrates a video/image decoding method according to an embodiment(s) of the present disclosure.

This disclosure is susceptible to various modifications and embodiments, and thus specific embodiments are illustrated and described in detail by way of illustration in the drawings. However, this is not intended to limit the embodiments of the disclosure to the specific embodiments. The terminology used herein is only used to describe specific embodiments, and is not intended to be limiting of the technical idea of the disclosure. The singular forms "a" and "an" as used herein are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term "and/or" as used herein includes any one or a combination of two or more of the associated listed items. The terms "comprises," "consists," and "contains" as used herein specify the presence of stated features, numbers, operations, elements, components, and/or combinations thereof, but do not preclude the presence or addition of one or more other features, numbers, operations, elements, components, and/or combinations thereof. The use of the term "can" or "may" in connection with an example or embodiment (e.g., what the example or embodiment can include or implement) in this disclosure means that there is at least one example or embodiment that includes or implements such feature, but not all examples are limited thereto and such feature or configuration may be omitted.

Meanwhile, each component in the drawings described in the present disclosure is independently depicted for the convenience of explaining different characteristic functions, and does not mean that each component is implemented with separate hardware or separate software. For example, two or more components among each component may be combined to form one component, or one component may be divided into multiple components. Embodiments in which each component is integrated and/or separated are also included in the scope of the present disclosure as long as they do not deviate from the essence of the present disclosure.

In this disclosure, "A or B" can mean "only A", "only B", or "both A and B". In other words, "A or B" in this disclosure can be interpreted as "A and/or B". For example, "A, B or C" in this disclosure can mean "only A", "only B", "only C", or "any combination of A, B and C".

The slash (/) or comma used in this disclosure can mean "and/or". For example, "A/B" can mean "A and/or B". Accordingly, "A/B" can mean "only A", "only B", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

In this disclosure, "at least one of A and B" can mean "only A", "only B" or "both A and B". Additionally, in this disclosure, the expressions "at least one of A or B" or "at least one of A and/or B" can be interpreted identically to "at least one of A and B".

Additionally, in the present disclosure, “at least one of A, B and C” can mean “only A,” “only B,” “only C,” or “any combination of A, B and C.” Additionally, “at least one of A, B or C” or “at least one of A, B and/or C” can mean “at least one of A, B and C.”

In addition, parentheses used in the present disclosure may mean "for example". Specifically, when it is indicated as "prediction (intra prediction)", "intra prediction" may be suggested as an example of "prediction". In other words, "prediction" in the present disclosure is not limited to "intra prediction", and "intra prediction" may be suggested as an example of "prediction". In addition, even when it is indicated as "prediction (i.e., intra prediction)", "intra prediction" may be suggested as an example of "prediction".

Technical features individually described in a single drawing in this disclosure may be implemented individually or simultaneously.

The present disclosure relates to video/image coding. For example, the method/embodiment described in the present disclosure can be applied to a method disclosed in the enhanced compression model (ECM) or H.267 standard. In addition, the method/embodiment disclosed in this disclosure can be applied to a method disclosed in the AV2 (AOMedia Video 2) standard, or a next-generation video/image coding standard (e.g. H.268, H.269, etc.).

In the present disclosure, coding may include encoding and/or decoding. In the present disclosure, image coding may be interchangeably used with video coding.

In the present disclosure, a video may mean a collection of a series of images over time. A picture generally means a unit representing one image of a specific time period, and a slice/tile is a unit that constitutes a part of a picture in coding. A slice/tile may include one or more CTUs (coding tree units). A picture may be composed of one or more slices/tiles. A tile may represent a rectangular area of CTUs within a specific tile row and a specific tile column within a picture.

Meanwhile, a picture may be divided into two or more subpictures. A subpicture may be a rectangular region of one or more slices within a picture.

A pixel or pel can mean the smallest unit that constitutes a picture (or image). Also, a 'sample' can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only the pixel/pixel value of the luma component, or only the pixel/pixel value of the chroma component.

A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (ex. cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as block or area. In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

Hereinafter, with reference to the attached drawings, embodiments of the present disclosure will be described in more detail. Hereinafter, the same reference numerals may be used for the same components in the drawings, and redundant descriptions of the same components may be omitted.

Figure 1 schematically illustrates an example of a video/image coding system to which embodiments of the present disclosure can be applied.

Referring to FIG. 1, a video/image coding system may include a first device (an encoding device) and a second device (a decoding device). The first device may transmit encoded video/image information or data to the second device through a digital storage medium or a network in the form of a file or streaming.

The video/image coding system may further include a video/image acquisition device and a video/image renderer. The video/image acquisition device may be included in the encoding device, or may be configured as a separate device or an external component. The video/image renderer may be included in the decoding device, or may be configured as a separate device or an external component.

The first device may include the transmission unit as an internal component, or as a separate device or external component.

The second device may include the receiver as an internal component, or may include it as a separate device or external component.

The encoder may be referred to as an encoding device, and the decoder may be referred to as a decoding device. The transmitting unit may be included in the encoding device. The receiving unit may be included in the decoding device. The renderer may include a display unit, and the display unit may be comprised of a separate device or an external component.

The decoding device and the encoding device to which the embodiment(s) of the present disclosure are applied may be included in a multimedia broadcasting transmitting/receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video conversation device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a video-on-demand (VoD) service providing device, an OTT video (Over the top video) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a VR (virtual reality) device, an AR (agumented reality) device, a video phone video device, a transportation terminal (ex. a vehicle (including an autonomous vehicle) terminal, an airplane terminal, a ship terminal, etc.), a medical video device, and the like, and may be used to process a video signal or a data signal. For example, the OTT video (Over the top video) device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, a DVR (Digital Video Recorder), and the like.

A video/image capture device can capture a video/image source. The video/image capture device can capture the video/image through a process of capturing, synthesizing, or generating the video/image. The video/image capture device can include a video/image capture device and/or a video/image generation device. The video/image capture device can include, for example, one or more cameras, a video/image archive containing previously captured video/image, etc. The video/image generation device can include, for example, a camcorder, a computer, a tablet, a smart phone, etc., and can (electronically) generate the video/image. For example, a virtual video/image can be generated through a computer, etc., in which case the video/image capture process can be replaced with a process of generating related data. The video/image source can also perform a video/image preprocessing process in order to input an optimized video/image to an encoder.

The encoding device can encode input video/image. The encoding device can perform a series of procedures such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video/image information) can be output in the form of a bitstream.

The transmission unit can transmit encoded video/image information or data output in the form of a bitstream to the reception unit of the receiving device through the network in the form of a file or streaming. The encoded video/image information or data output in the form of a bitstream can also be transmitted to the reception unit through a streaming server. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit can include an element for generating a media file through a predetermined file format and can include an element for transmission through a broadcasting/communication network. The reception unit can receive/extract the bitstream and transmit it to a decoding device.

The streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream. The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as a medium that informs the user of any available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server transmits multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server serves to control commands/responses between each device within the content streaming system.

The above streaming server can receive content from a media storage and/or an encoding device. For example, when receiving content from the encoding device, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

The decoding device can decode the video/image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding device.

The renderer can render the decoded video/image. The rendered video/image can be displayed through the display unit.

FIG. 2 is a drawing schematically illustrating the configuration of a video/image encoding device to which embodiments of the present disclosure may be applied. The encoding device hereinafter may include an image encoding device and/or a video encoding device.

Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a prediction unit (predictor) 220, a residual processor (residual processor) 230, an entropy encoder (entropy encoder) 240, an adder (adder) 250, a filter (filter) 260, and a memory (memory) 270. The prediction unit (220) may include an inter prediction unit and an intra prediction unit. The residual processor (230) may include a transformer (transformer) 232, a quantizer (quantizer) 233, a dequantizer (dequantizer) 234, and an inverse transformer (inverse transformer) 235. The residual processor (230) may further include a subtractor (subtractor) 231. The addition unit (250) may be called a reconstructor or a reconstructed block generator. The image segmentation unit (210), the prediction unit (220), the residual processing unit (230), the entropy encoding unit (240), the addition unit (250), and the filtering unit (260) described above may be configured by one or more hardware components (e.g., an encoder chipset or a processor) according to an embodiment. In addition, the memory (270) may include a DPB (decoded picture buffer) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal/external component.

The image segmentation unit (210) can segment an input image (or, picture, frame) input to the encoding device (200) into one or more processing units. For example, the processing unit may be called a coding unit (CU). In this case, the coding unit may be recursively segmented from a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, one coding unit may be segmented into a plurality of coding units of deeper depth based on a quad-tree structure, a binary tree structure, and/or a ternary structure. In this case, for example, the quad-tree structure may be applied first and the binary tree structure and/or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied first. The coding procedure according to the present disclosure may be performed based on a final coding unit that is no longer segmented. In this case, based on coding efficiency according to image characteristics, etc., the maximum coding unit can be used as the final coding unit, or, if necessary, the coding unit can be recursively divided into coding units of lower depths, and the coding unit of the optimal size can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be divided or partitioned from the final coding unit described above, respectively. The prediction unit may be a unit for sample prediction, and the transformation unit may be a unit for deriving a transform coefficient and/or a unit for deriving a residual signal from a transform coefficient.

The term unit may be used interchangeably with terms such as block or area, depending on the case. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component, or only a pixel/pixel value of a chroma component. A sample can be used as a term corresponding to a pixel or pel in a picture (or image).

The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from a prediction unit from an input image signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, as illustrated, a unit that subtracts a prediction signal (prediction block, prediction sample array) from an input image signal (original block, original sample array) within the encoder (200) may be called a subtraction unit (231). The prediction unit can perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including prediction samples for the current block. The prediction unit can determine whether intra prediction or inter prediction is applied per current block or CU unit. The prediction unit can generate various information about prediction, such as prediction mode information, as described later in the description of each prediction mode, and transmit it to the entropy encoding unit (240). The information about prediction can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

The intra prediction unit can predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from it depending on the prediction mode. In the intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the degree of detail of the prediction direction. However, this is only an example, and a number of directional prediction modes more or less than that may be used depending on the setting. The intra prediction unit may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

An inter prediction unit can derive a predicted block for a current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in an inter prediction mode, the motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include information on an inter prediction direction (such as L0 prediction, L1 prediction, or Bi prediction). In the case of inter prediction, neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. A reference picture including the reference block and a reference picture including the temporal neighboring blocks may be the same or different. The above temporal neighboring blocks may be called collocated reference blocks, collocated CUs (colCUs), etc., and a reference picture including the temporal neighboring blocks may be called a collocated picture (colPic). For example, the inter prediction unit may configure a motion information candidate list based on the neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and/or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of the skip mode and the merge mode, the inter prediction unit may use the motion information of the neighboring blocks as the motion information of the current block. In the case of the skip mode, unlike the merge mode, a residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of the neighboring blocks may be used as a motion vector predictor, and the motion vector of the current block may be indicated by signaling a motion vector difference.

The prediction unit (220) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit can apply intra prediction or inter prediction for prediction of one block, and can also apply intra prediction and inter prediction at the same time. This can be called combined inter and intra prediction (CIIP). In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for SCC (screen content coding), for example. IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block based on a block vector within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described in the present disclosure.

The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or to generate a residual signal. The transform unit (232) can apply a transform technique to the residual signal to generate transform coefficients. For example, the transform technique can include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loe've Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT).

The quantization unit (233) quantizes the transform coefficients and transmits them to the entropy encoding unit (240), and the entropy encoding unit (240) can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients can be called residual information. The quantization unit (233) can rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and can also generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoding unit (240) can perform various encoding methods, such as, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoding unit (240) may encode, together or separately, information necessary for video/image restoration (e.g., values of syntax elements, etc.) in addition to quantized transform coefficients. The encoded information (e.g., encoded video/image information) may be transmitted or stored in the form of a bitstream in the form of a network abstraction layer (NAL) unit. The video/image information may further include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general constraint information. In the present disclosure, information and/or syntax elements transmitted/signaled from an encoding device to a decoding device may be included in the video/image information. The video/image information may be encoded through the above-described encoding procedure and included in the bitstream. The bitstream may be transmitted through a network or stored in a digital storage medium. Here, the network may include a broadcasting network and/or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit (240) may be configured as an internal/external element of the encoding device (200) by a transmitting unit (not shown) and/or a storing unit (not shown), or the transmitting unit may be included in the entropy encoding unit (240).

The quantized transform coefficients output from the quantization unit (233) can be used to generate a prediction signal. For example, by applying inverse quantization and inverse transformation to the quantized transform coefficients through the inverse quantization unit (234) and the inverse transform unit (235), a residual signal (residual block or residual samples) can be reconstructed. The addition unit (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the prediction unit. When there is no residual for the target block to be processed, such as when the skip mode is applied, the predicted block can be used as the reconstructed block. The addition unit (250) can be called a reconstructed unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next target block in the current picture, and can also be used for inter prediction of the next picture after filtering as described below.

Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and/or restoration process.

The filtering unit (260) can apply filtering to the restoration signal to improve subjective/objective picture quality. For example, the filtering unit (260) can apply various filtering methods to the restoration picture to generate a modified restoration picture and store the modified restoration picture in the memory (270), specifically, in the DPB of the memory (270). The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate information about the filtering and transmit it to the entropy encoding unit (240). The information about the filtering can be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter prediction unit. Through this, the encoding device can avoid prediction mismatch between the encoding device (200) and the decoding device when inter prediction is applied, and can also improve encoding efficiency.

The memory (270) DPB can store the modified restored picture to be used as a reference picture in the inter prediction unit. The memory (270) can store motion information of a block from which motion information in the current picture is derived (or encoded) and/or motion information of blocks in a picture that has already been restored. The stored motion information can be transferred to the inter prediction unit to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (270) can store restored samples of restored blocks in the current picture and transfer them to the intra prediction unit.

FIG. 3 is a drawing schematically illustrating the configuration of a video/image decoding device to which embodiments of the present disclosure may be applied. The decoding device hereinafter may include an image decoding device and/or a video decoding device.

Referring to FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter prediction unit and an intra prediction unit. The residual processor (320) may include a dequantizer (321) and an inverse transformer (321). The entropy decoding unit (310), residual processing unit (320), prediction unit (330), adding unit (340), and filtering unit (350) described above may be configured by one hardware component (e.g., decoder chipset or processor) according to an embodiment. In addition, the memory (360) may include a DPB (decoded picture buffer) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal/external component.

When a bitstream including video/image information is input, the decoding device (300) can restore the image corresponding to the process in which the video/image information is processed in the encoding device of FIG. 2. For example, the decoding device (300) can derive units/blocks based on block division related information obtained from the bitstream. The decoding device (300) can perform decoding using a processing unit applied in the encoding device. Therefore, the processing unit of decoding can be, for example, a coding unit, and the coding unit can be divided from a coding tree unit or a maximum coding unit according to a quad tree structure, a binary tree structure, and/or a ternary tree structure. One or more transform units can be derived from the coding unit. Then, the restored image signal decoded and output by the decoding device (300) can be reproduced through a reproduction device.

The decoding device (300) can receive a signal output from the encoding device in the form of a bitstream, and the received signal can be decoded through the entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information (e.g., video/image information) necessary for image restoration (or picture restoration). The video/image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general constraint information. The decoding device can decode the picture further based on the information on the parameter set and/or the general constraint information. The signaling/received information and/or syntax elements described later in the present disclosure can be obtained from the bitstream by being decoded through the decoding procedure. For example, the entropy decoding unit (310) can decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output values of syntax elements required for image restoration and quantized values of transform coefficients for residuals. More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntax element in the bitstream, determines a context model by using information of a syntax element to be decoded and decoding information of surrounding and decoding target blocks or information of symbols/bins decoded in a previous step, and predicts an occurrence probability of a bin according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element. At this time, the CABAC entropy decoding method can update the context model by using information of the decoded symbol/bin for the context model of the next symbol/bin after determining the context model. Among the information decoded by the entropy decoding unit (310), information regarding prediction is provided to the prediction unit (330), and the residual values on which entropy decoding is performed by the entropy decoding unit (310), that is, quantized transform coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive a residual signal (residual block, residual samples, residual sample array). In addition, information regarding filtering among the information decoded by the entropy decoding unit (310) can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal/external element of the decoding device (300), or the receiving unit may be a component of the entropy decoding unit (310). Meanwhile, the decoding device according to the present disclosure may be called a video/video/picture decoding device, and the decoding device may be divided into an information decoder (video/video/picture information decoder) and a sample decoder (video/video/picture sample decoder). The information decoder may include the entropy decoding unit (310), and the sample decoder may include at least one of the inverse quantization unit (321), the inverse transformation unit (322), the adding unit (340), the filtering unit (350), the memory (360), and the prediction unit (330).

The inverse quantization unit (321) can inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit (321) can rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device. The inverse quantization unit (321) can perform inverse quantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain transform coefficients.

In the inverse transform unit (322), the transform coefficients are inversely transformed to obtain a residual signal (residual block, residual sample array).

The prediction unit can perform a prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit can determine whether intra prediction or inter prediction is applied to the current block based on the information about the prediction output from the entropy decoding unit (310), and can determine a specific intra/inter prediction mode.

The prediction unit (330) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit can apply intra prediction or inter prediction for prediction of one block, and can also apply intra prediction and inter prediction at the same time. This can be called combined inter and intra prediction (CIIP). In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for SCC (screen content coding), for example. IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block based on a block vector within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described in the present disclosure.

The intra prediction unit can predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from it depending on the prediction mode. In the intra prediction, the prediction modes may include multiple non-directional modes and multiple directional modes. The intra prediction unit may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

The inter prediction unit can derive a predicted block for a current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include information on an inter prediction direction (such as L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) can configure a motion information candidate list based on neighboring blocks, and derive a motion vector and/or a reference picture index of the current block based on the received candidate selection information. Inter prediction can be performed based on various prediction modes, and information about the prediction can include information indicating a mode of inter prediction for the current block.

The addition unit (340) can generate a restoration signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the prediction signal (predicted block, predicted sample array) output from the prediction unit (330). When there is no residual for the target block to be processed, such as when skip mode is applied, the predicted block can be used as the restoration block.

The addition unit (340) may be called a restoration unit or a restoration block generation unit. The generated restoration signal may be used for intra prediction of the next processing target block within the current picture, may be output after filtering as described below, or may be used for inter prediction of the next picture.

Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

The filtering unit (350) can apply filtering to the restoration signal to improve subjective/objective image quality. For example, the filtering unit (350) can apply various filtering methods to the restoration picture to generate a modified restoration picture, and transmit the modified restoration picture to the memory (360), specifically, the DPB of the memory (360). The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

The (corrected) reconstructed picture stored in the DPB of the memory (360) can be used as a reference picture in the inter prediction unit. The memory (360) can store motion information of a block from which motion information in the current picture is derived (or decoded) and/or motion information of blocks in a picture that has already been reconstructed. The stored motion information can be transferred to the inter prediction unit to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (360) can store reconstructed samples of reconstructed blocks in the current picture and transfer them to the intra prediction unit.

In this specification, the embodiments described in the filtering unit (260) and the prediction unit (220) of the encoding device (200) can be applied identically or correspondingly to the filtering unit (350) and the prediction unit (330) of the decoding device (300).

As described above, prediction is performed in order to increase compression efficiency when performing video coding. Through this, a predicted block including prediction samples for a current block, which is a coding target block, can be generated. Here, the predicted block includes prediction samples in a spatial domain (or pixel domain). The predicted block is derived identically from an encoding device and a decoding device, and the encoding device can increase image coding efficiency by signaling information (residual information) about a residual between the original block and the predicted block, rather than the original sample value of the original block itself, to a decoding device. The decoding device can derive a residual block including residual samples based on the residual information, and generate a reconstructed block including reconstructed samples by combining the residual block and the predicted block, and can generate a reconstructed picture including the reconstructed blocks.

The residual information can be generated through a transformation and quantization procedure. For example, the encoding device can derive a residual block between the original block and the predicted block, perform a transformation procedure on residual samples (residual sample array) included in the residual block to derive transform coefficients, perform a quantization procedure on the transform coefficients to derive quantized transform coefficients, and signal related residual information to a decoding device (via a bitstream). Here, the residual information can include information such as value information of the quantized transform coefficients, position information, transformation technique, transformation kernel, and quantization parameter. The decoding device can perform an inverse quantization/inverse transformation procedure based on the residual information to derive residual samples (or residual block). The decoding device can generate a reconstructed picture based on the predicted block and the residual block. The encoding device can also inversely quantize/inversely transform the quantized transform coefficients to derive a residual block for reference in inter prediction of a subsequent picture, and generate a restored picture based on the residual block.

In the present disclosure, at least one of quantization/dequantization and/or transform/inverse transform may be omitted. If the quantization/dequantization is omitted, the quantized transform coefficient may be called a transform coefficient. If the transform/inverse transform is omitted, the transform coefficient may be called a coefficient or a residual coefficient, or may still be called a transform coefficient for the sake of uniformity of expression.

In addition, in the present disclosure, the quantized transform coefficients and the transform coefficients may be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, the residual information may include information about the transform coefficient(s), and the information about the transform coefficient(s) may be signaled via residual coding syntax. Transform coefficients may be derived based on the residual information (or the information about the transform coefficient(s)), and scaled transform coefficients may be derived through inverse transformation (scaling) on the transform coefficients. Residual samples may be derived based on inverse transformation (transformation) on the scaled transform coefficients. This may be similarly applied/expressed in other parts of the present disclosure.

Intra prediction may represent a prediction that generates prediction samples for a current block based on reference samples in a picture to which the current block belongs (hereinafter, the current picture). When intra prediction is applied to the current block, peripheral reference samples to be used for intra prediction of the current block may be derived. The peripheral reference samples of the current block may include H+W samples located at the left of a current block of a size WХH, W+H samples located at the top of the current block, and at least one sample neighboring the top-left of the current block. Alternatively, the peripheral reference samples of the current block may include upper peripheral samples of multiple rows and left peripheral samples of multiple columns.

Some of the surrounding reference samples of the current block may not be decoded yet or may not be available. In this case, the decoder can construct surrounding reference samples to be used for prediction by padding or substituting the unavailable samples with the available samples.

When the surrounding reference samples are derived, the prediction samples of the current block can be derived based on the surrounding reference samples and the intra prediction mode/type information. Here, the intra prediction mode can represent one of the non-directional prediction modes and the directional prediction modes that indicate spatial correlation for intra prediction. Here, the directional prediction mode can be called an angular prediction mode, and the non-directional prediction mode can be called a non-angular prediction mode. The intra prediction type can represent various prediction types for performing intra prediction. The intra prediction type may include, for example, MRL (multi-reference line), ISP (intra sub-partitions), PDPC (Position dependent intra prediction), MIP (matrix weighted intra prediction or matrix based intra prediction), CCLM (cross-component linear model), MMLM (multi-model linear model), DIMD (Decoder side intra mode derivation), fusion of chroma intra prediction modes, intra template matching, TIMD (fusion for template-based intra mode derivation), intra prediction fusion, CCCM (cross-component convolutional model), CCP (cross-component prediction), SGPM (spatial geometric partitioning mode), etc. In some cases, the intra prediction mode and/or the intra prediction type may be used to perform intra prediction.

Specifically, the intra prediction procedure may include an intra prediction mode/type determination step, a reference sample derivation step, and an intra prediction mode/type-based prediction sample derivation step. In addition, a post-filtering step may be performed on the derived prediction sample as needed.

Figure 4 illustrates an intra prediction procedure as an example.

Referring to FIG. 4, the intra prediction procedure as described above may include an intra prediction mode/type determination step, a reference sample derivation step, and an intra prediction performance (prediction sample generation) step. The intra prediction procedure may be performed in an encoding device and a decoding device as described above.

The coding device determines the intra prediction mode/type (S400). The coding device may include an encoding device and/or a decoding device as described above.

An encoding device can determine an intra prediction mode/type applied to the current block from among various intra prediction modes/types described in the present disclosure, and can generate prediction-related information. The prediction-related information can include intra prediction mode information indicating an intra prediction mode applied to the current block and/or intra prediction type information indicating an intra prediction type applied to the current block. A decoding device can determine an intra prediction mode/type applied to the current block based on the prediction-related information.

For example, when intra prediction is applied, the intra prediction mode to be applied to the current block can be determined using the intra prediction mode of the surrounding block. For example, the coding device can select one of the MPM candidates in the MPM (most probable mode) list derived based on the intra prediction mode of the surrounding block (e.g., the left and/or upper surrounding block) of the current block and/or additional candidate modes based on the received index information, or can select one of the remaining intra prediction modes not included in the MPM candidates based on MPM remind information (remaining intra prediction mode information). The MPM list can be configured to include or not include the planar mode as a candidate.

The coding device can construct an MPM (most probable modes) list for the current block. The MPM list may also be expressed as an MPM candidate list. Here, MPM may mean a mode used to improve coding efficiency by considering the similarity between the current block and surrounding blocks during intra prediction mode coding.

The encoding device can perform prediction based on various intra prediction modes, and determine an optimal intra prediction mode based on RDO (rate-distortion optimization) based on the prediction. In this case, the encoding device can determine the optimal intra prediction mode using MPM candidates configured in the MPM list, or can determine the optimal intra prediction mode using the remaining intra prediction modes other than the MPM candidates configured in the MPM list. Specifically, for example, if the intra prediction type of the current block is not a normal intra prediction type but a specific type (e.g., DIMD, TIMD, MRL or ISP), the encoding device can determine the optimal intra prediction mode by considering only the MPM candidates as intra prediction mode candidates for the current block. That is, in this case, the intra prediction mode for the current block can be determined only among the MPM candidates, and in this case, the MPM flag may not be encoded/signaled. In this case, the decoding device can estimate that the MPM flag is 1 without being separately signaled the MPM flag.

Meanwhile, in general, if the intra prediction mode of the current block is not a planar mode but one of the MPM candidates in the MPM list, the encoding device generates an mpm index (mpm idx) pointing to one of the MPM candidates. If the intra prediction mode of the current block is not in the MPM list either, the encoding device generates MPM reminder information (remaining intra prediction mode information) pointing to a mode same as the intra prediction mode of the current block among the remaining intra prediction modes not included in the MPM list (and the planar mode). The MPM reminder information may include, for example, an intra_luma_mpm_remainder syntax element.

A decoding device obtains intra prediction mode information from a bitstream. The intra prediction mode information may include at least one of an MPM flag, an MPM index, and MPM remind information (remaining intra prediction mode information) as described above. The decoding device may configure an MPM list. The MPM list is configured in the same manner as the MPM list configured in the encoding device. That is, the MPM list may include intra prediction modes of surrounding blocks, and may further include specific intra prediction modes according to a predetermined method.

The decoding device can determine the intra prediction mode of the current block based on the MPM list and the intra prediction mode information. For example, when the value of the MPM flag is 1, the decoding device can derive a candidate indicated by the MPM index among the MPM candidates in the MPM list as the intra prediction mode of the current block.

As another example, when the value of the MPM flag is 0, the decoding device can derive the intra prediction mode indicated by the remaining intra prediction mode information (which may be called mpm remainder information) from among the remaining intra prediction modes as the intra prediction mode of the current block.

The coding device derives reference samples of the current block (S410). The reference samples may include peripheral reference samples of the current block. The peripheral reference samples of the current block may include H+W samples located at the left of a current block of a size WХH, W+H samples located at the top of the current block, and at least one sample neighboring the top-left of the current block. Alternatively, the peripheral reference samples of the current block may include upper peripheral samples of multiple rows and left peripheral samples of multiple columns.

The coding device performs intra prediction on the current block to derive prediction samples (S1220). The coding device can derive the prediction samples based on the intra prediction mode/type and the reference samples. The coding device can derive a reference sample according to the intra prediction mode of the current block among the reference samples of the current block, and derive a prediction sample of the current block based on the reference sample.

An encoding procedure based on intra prediction may roughly include, for example:

Figure 5 shows examples of intra prediction based video/image encoding methods.

Referring to FIG. 5, S500 may be performed by a prediction unit of an encoding device, S505 may be performed by a residual processing unit of the encoding device, and S510 or S515 may be performed by an entropy encoding unit of the encoding device. Specifically, the prediction-related information may be derived by the prediction unit and encoded by the entropy encoding unit. The residual information may be derived by the residual processing unit and encoded by the entropy encoding unit. The residual information is information about the residual samples. The residual information may include information about quantized transform coefficients for the residual samples. As described above, the residual samples may be derived as transform coefficients through the transform unit of the encoding device, and the transform coefficients may be derived as quantized transform coefficients through the quantization unit. The information about the quantized transform coefficients may be encoded in the entropy encoding unit through a residual coding procedure.

The encoding device performs intra prediction on the current block (S500). The encoding device can derive an intra prediction mode/type for the current block, derive reference samples of the current block, and generate prediction samples in the current block based on the intra prediction mode/type and the reference samples. Here, the intra prediction mode/type determination, surrounding reference sample derivation, and prediction sample generation procedures may be performed simultaneously, or one procedure may be performed before the other procedure. The encoding device can determine a mode/type applied to the current block among a plurality of intra prediction modes/types. The encoding device can compare RD costs for the intra prediction modes/types and determine an optimal intra prediction mode/type for the current block.

Meanwhile, the encoding device may also perform a prediction sample filtering procedure. The prediction sample filtering may be referred to as post-filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.

The encoding device generates residual samples for the current block based on the prediction samples (S505). The encoding device can compare the prediction samples with the original samples of the current block based on phase and derive the residual samples.

The encoding device can encode image/video information including information about the intra prediction (prediction-related information) and/or information about the residual samples (residual information) (S510 or S515). The prediction-related information can include the intra prediction mode information and the intra prediction type information. The encoding device can output the encoded image/video information in the form of a bitstream. The output bitstream can be transmitted to a decoding device via a storage medium or a network.

The residual information may include a residual coding syntax element. An encoding device may transform/quantize the residual samples to derive quantized transform coefficients. The residual information may include information about the quantized transform coefficients.

Meanwhile, as described above, the encoding device can generate a restored picture (including restored samples and restored blocks). To this end, the encoding device can perform inverse quantization/inverse transformation on the quantized transform coefficients again to derive (corrected) residual samples. The reason for performing inverse quantization/inverse transformation on the residual samples after transforming/quantizing them in this way is to derive residual samples that are identical to the residual samples derived from the decoding device as described above. The encoding device can generate a restored block including restored samples for the current block based on the prediction samples and the (corrected) residual samples. A restored picture for the current picture can be generated based on the restored block. As described above, an in-loop filtering procedure, etc. can be further applied to the restored picture.

The decoding device can perform operations corresponding to the operations performed in the encoding device. A video/image decoding procedure based on intra prediction may include, for example, the following.

Figure 6 shows examples of intra prediction based video/image decoding methods.

Referring to FIG. 6, S600 may be performed by an entropy decoding unit of a decoding device, S610 may be performed by a prediction unit of the decoding device, S615 may be performed by a residual processing unit of the decoding device, and S620 may be performed by an adder or restoration unit of the decoding device.

Specifically, the decoding device obtains image/video information from the bitstream (S600). The image/video information may include prediction-related information and/or residual information.

The decoding device performs intra prediction based on prediction-related information (S610). The decoding device may derive an intra prediction mode/type for a current block based on the prediction-related information, derive reference samples of the current block, and generate prediction samples in the current block based on the intra prediction mode/type and the reference samples. In this case, the decoding device may perform a prediction sample filtering procedure. The prediction sample filtering may be referred to as post filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.

The decoding device performs residual processing based on the residual information (S615). The decoding device can derive residual samples for the current block based on the residual information. Specifically, the inverse quantization unit of the residual processing unit performs inverse quantization based on the quantized transform coefficients derived based on the residual information to derive transform coefficients, and the inverse transform unit of the residual processing unit can perform inverse transformation on the transform coefficients to derive residual samples for the current block.

The decoding device generates a restored block/picture (S620). The decoding device can generate restored samples for the current block based on the prediction samples and/or the residual samples, and derive a restored block including the restored samples. A restored picture for the current picture can be generated based on the restored block. As described above, an in-loop filtering procedure, etc. can be further applied to the restored picture.

The intra prediction mode information and/or the intra prediction type information can be encoded/decoded through the binarization and coding method described in the present disclosure. For example, the intra prediction mode information and/or the intra prediction type information can be binarized through fixed-length binarization, truncated Rice binarization, truncated unary binarization, etc. For example, the intra prediction mode information and/or the intra prediction type information can be encoded/decoded through entropy coding (ex. CABAC, CAVLC) coding.

Fig. 7 shows examples of directional intra prediction modes. Fig. 7 can show an example of a case including 65 directional intra prediction modes.

Referring to FIG. 7, the directional intra prediction modes may include directional intra prediction modes of mode #2 to mode #65. In this case, mode #50 may represent a vertical intra prediction mode, and mode #18 may represent a horizontal intra prediction mode. Meanwhile, this is just an example, and the number and types of candidate intra prediction modes may be changed. Meanwhile, one or more non-directional intra prediction modes may be considered, for example, mode #0 may represent an intra planar prediction mode (planner mode), and mode #1 may represent an intra DC prediction mode (DC mode).

For intra prediction modes, mode numbers can be assigned, for example, as shown in the following table.

Intra prediction mode Associated name 0 INTRA_PLANAR 1 INTRA_DC 2...66 INTRA_ANGULAR2...INTRA_ANGULAR66 81...83 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

According to the present disclosure, various intra prediction modes can be used for intra prediction of a current block. For example, an intra prediction mode can be derived based on a DIMD (Decoder side intra mode derivation) technique. DIMD can be referred to as a DIMD type.

Figure 8 shows an example of template-based HoG computation in DIMD.

Referring to Fig. 8, in DIMD, a Histogram of Gradient (HoG) can be calculated using a template that includes surrounding samples of a current block. For example, a HoG can be calculated using a horizontal/vertical Sobel filter based on a line template of n (ex. 3) surrounding samples. In this case, horizontal and vertical Sobel filters can be applied to calculate the HoG. If the template is located in a different CTU, the Sobel filter may not be applied to the upper CTU boundary, or the DIMD technique may not be applied to the current block.

Meanwhile, the filter applied for HoG calculation may be determined differently based on the block size. For example, when the block size is 4x4, 4x8, or 8x4, a 2x2 kernel filter may be applied instead of a 3x3 Sobel filter for HoG calculation.

Meanwhile, an adaptive number of (surrounding) reference samples can be used for the HoG calculation of DIMD. In this case, the number of sample lines in the DIMD template can be 4 or more.

For the candidate intra prediction modes, n intra prediction modes having the highest histogram values can be selected/extracted through the HoG calculation. In this case, the prediction block for the current block can be derived using the n intra prediction modes. In addition, in this case, the prediction block can be derived using the n intra prediction modes and a non-directional mode (e.g., planar mode or DC mode). In this case, a predictor according to each intra prediction mode can be derived, and the prediction block can be derived by weighting and summing/weighting averaging them. In other words, the predictors of the n (e.g., 5) selected/extracted intra prediction modes and the predictor of the planar mode can be fused through prediction fusion.

For example, the above n can be 5. In this case, five derived modes with the highest HoG can be derived. The derived mode can be called a DIMD-based mode or a DIMD derived mode. For example, the number of the above n can be determined differently based on the block size. For example, when the W*H of the block is 128 or more, n can be 7, and in other cases, n can be 5.

Meanwhile, for the HoG calculation of DIMD, a specific block vector may be used. For example, the block vector may be derived based on a motion vector or a block vector of a surrounding block. A reference area of a moved position may be derived using a specific block vector based on the position of the current block, and the HoG may be calculated using (restored) reference samples within the reference area. In this case, the template may be located in the reference area or may be located on the upper/left periphery of the reference area. In this case, the size of the reference area may be the same as the size of the current block. Meanwhile, the size of the reference area may be different from the size of the current block. For example, the width of the reference area may be half of the width of the current block, and/or the height of the reference area may be half of the height of the current block. In this case, the size of the template may also be changed within the range of the width and height of the reference area.

The n intra modes derived through the above DIMD can be called DIMD derived modes or DIMD-based modes. The above DIMD derived modes can be included as candidates in the above MPM list (e.g. PMPM list).

When the DIMD technique (or type) is applied to the current block, the first intra mode derived by the DIMD can be stored as the intra mode of the target block and can be referenced in configuring the MPM list of the subsequent block.

When the DIMD technique (or type) is applied to the current block, the first intra mode derived by the DIMD can be referred to as the mode for selecting the transformation kernel (or transformation set).

Meanwhile, for the above weighted sum/weighted average or prediction fusion, weights between predictors are required. According to an embodiment of the present disclosure, a look-up table (LUT) can be used to derive weights. While calculating dimdMode _i using the existing full HoG, Habove and Hleft can be calculated separately to calculate whether dimdMode _i is highly dependent on the upper template or the left template, that is, whether it is location-dependent.

If it is not position dependent, the weights (wDimd _i , wPlanar) can be determined based on the HoG size as before.

If position-dependent, sample-based blending can be applied, in which case weights can be applied differently on a sample-by-sample basis.

If the upper HoG (ex. Habove) is equal to or greater than twice the left HoG (ex. Hleft), the weights can be based on the following mathematical formula:

Here, Δ _i can represent the difference or its absolute value between the upper HoG and the left HoG.

If the left HoG (ex. Hleft) is equal to or greater than twice the upper HoG (ex. Habove), the weights can be based on the following mathematical formula:

Here, Δ _i can represent the difference or its absolute value between the left HoG and the upper HoG.

Meanwhile, additional W columns can be used if the upper right side is available as surrounding reference samples for DIMD, and additional H rows can be used if the lower left side is available. That is, the region of the template containing surrounding (reconstruction) samples for HOG computation can be based on the availability of the surrounding (reconstruction) samples.

Meanwhile, if n intra (prediction) modes are derived from DIMD, n intra mode predictors and planner predictors are combined, but if one intra mode is derived from DIMD, only the one intra mode can be used. Or, even if one intra mode is derived from DIMD, it can be combined with the planner mode predictor. For example, if the HoG value of the intra mode having the highest HoG in DIMD is equal to or greater than a threshold value, only the one intra mode can be derived. As another example, if the first HoG value of the intra mode having the highest HoG in DIMD is greater than the second HoG value of the intra mode having the second highest HoG by a predetermined value or more, only the one intra mode can be derived. In addition, n intra prediction modes having HoGs greater than a predetermined threshold value among the DIMD derived modes may be selected.

Meanwhile, for a block to which DIMD is applied, the first mode derived from DIMD can be stored as the intra prediction mode of the current block. The stored mode can be referenced as a mode of a surrounding block and/or a mode for selecting a transformation kernel (or transformation set) to be referenced for deriving the intra prediction mode of a subsequent block.

Meanwhile, there may be cases where the DIMD-derived first mode is not the same as the actual prediction mode of the current block (or the optimal prediction mode most correlated with the current block). Therefore, it is necessary to review modes that can be stored as the intra prediction mode of the current block other than the first intra prediction mode derived by DIMD. For example, if the HoG of the DIMD-derived first mode is greater than a certain threshold, the first mode can be stored as the intra prediction mode of the current block, otherwise, the planner or DC mode can be stored as the intra prediction mode of the current block. Alternatively, if the first predictor (first prediction samples) of the current block generated by DIMD and the second predictor (second prediction samples) generated by the DIMD-derived first mode are compared and are within a certain SAD/SATD, the DIMD-derived first mode can be stored as the intra prediction mode of the current block, otherwise, the planner or DC mode can be stored as the intra prediction mode of the current block.

As another example, a candidate list may be constructed with m intra prediction modes derived by DIMD, and one of the candidate lists may be stored. Here, m may be equal to n, or m may be 2 or 3. In this case, the first predictor (first prediction samples) of the current block generated by DIMD may be compared with predictors according to each of the m intra prediction modes, and the intra prediction mode having the lowest SAD/SATD among the m intra prediction modes may be stored as the intra prediction mode of the current block.

Alternatively, for the first predictor (first prediction samples) derived by the DIMD technique, the first intra prediction mode derived based on the DIMD technique (i.e., the intra prediction mode (the first intra prediction mode) having the highest HoG based on the Sobel filter-based HoG derived from the prediction samples) may be stored as the intra prediction mode of the current block. In this case, the first intra prediction mode derived based on the template of the current block and the first intra prediction mode derived based on the predictor of the current block may be different.

For the above DIMD, DIMD related information may be signaled. The DIMD related information may include at least one of a DIMD availability flag, a DIMD application flag, and/or a DIMD flag.

The DIDM enabled flag (dimd_enabeld_flag) can be signaled in higher level syntax (e.g. SPS). When the value of the DIMD enabled flag is 1, the DIMD applied flag (dimd_applied_flag) and/or the DIMD flag (dimd_flag) can be signaled.

The above DIMD application flag can be signaled at the PPS, PH (picture header), SH (slice header) or CTU level. Even when DIMD is available, DIMD can be turned on/off at the picture, slice or CTU level through the DIMD application flag. The DIMD flag can be signaled at the CU, PU or TU level. The DIMD flag can be signaled before the MPM flag (or PMPM flag).

Signaling of the above DIMD related information may include, for example:

The DIMD flag (dimd_flag) can be signaled before the PMPM flag (pmpm_flag). In this case, the PMPM flag is signaled when the value of the DIMD flag is 0, and signaling can be omitted when the value of the DIMD flag is 1. This can be expressed, for example, as shown in the following table.

Meanwhile, the DIMD flag (dimd_flag) can be signaled before (in the parsing order) the reference line index (intra_luma_ref_idx). The reference line index is information indicating one or more of a plurality of surrounding reference sample lines when the plurality of surrounding reference sample lines are used for intra prediction of the current block. The reference line index can be signaled when the DIMD flag is not 1. The PMPM flag can be signaled when the DIMD flag is not 1 and the reference sample line index indicates 0. This can be expressed, for example, as shown in the following table.

Meanwhile, the DIMD flag (dimd_flag) can be signaled after the reference line index (in the parsing order) (in the intra_luma_ref_idx). The DIMD flag can be signaled when the value of the reference line index is 0. The PMPM flag can be signaled when the DIMD flag is not 1 and the reference sample line index points to 0. This can be expressed, for example, as shown in the following table.

Meanwhile, as described above, an MPM list may be configured to efficiently signal the intra prediction mode of the current block. In a method according to one embodiment of the present disclosure, a plurality of MPM lists may be configured. The plurality of MPM lists may include a first MPM list and a second MPM list. The first MPM list may be called a PMPM (primary MPM) list, and the second MPM list may be called an SMPM (secondary MPM) list.

The above first MPM list and the second MPM list can be configured according to predetermined criteria. For example, a general MPM list including k candidates may be configured first, and the first n candidates of the general MPM list may be included in the first MPM list, and the remaining m candidates may be included in the second MPM list. As an example, k may be 22, n may be 6, and/or m may be 16. As an example, the first candidate (the candidate of the first entry) of the general MPM list may always be the planar mode. As another example, when the planar mode is signaled based on a separate planar flag (or not_planar_flag), k may be 21, n may be 5, and/or m may be 16.

At least one of the remaining candidates, excluding the planar mode, can be derived from the surrounding blocks.

Figure 9 shows examples of peripheral blocks for deriving the MPM list.

Referring to FIG. 9, the surrounding blocks may include at least one of a left surrounding block (L), a lower left surrounding block (BL), an upper surrounding block (A), an upper right surrounding block (AR), and/or an upper left surrounding block (AL) of the current block.

In addition, the remaining candidates may include DIMD-based modes (intra prediction modes derived based on DIMD). For example, intra prediction modes derived from surrounding blocks and DIMD-based modes (intra prediction modes derived based on DIMD) may be sorted in order of decreasing SAD cost based on SAD cost (hereinafter, referred to as sorted intra prediction modes) and added to the general MPM list or the first MPM list. In this case, only a predetermined number p of sorted intra prediction modes may be extracted and added to the general MPM list or the first MPM list. Here, the SAD cost may be calculated based on a predictor derived by applying the candidate mode to the restored samples of the template of the current block and the template. Directional prediction modes among the sorted intra prediction modes may be referred to as sorted directional modes. The p may be, for example, 5 or 6. Alternatively, the p may be 7 or 8. In the above DIMD-based modes, the planar mode and/or the DC mode may be excluded. The same applies hereinafter. The DIMD-based mode may be called a DIMD derivation mode.

For example, one could add the sorted prediction modes, modes with offsets added or subtracted from the (extracted) sorted directional modes, and some default modes until all k entries are filled.

As another example, the sorting procedure may be omitted for some or all of the DIMD-based modes. This is because the DIMD-based modes have a high correlation with the current block. Therefore, the sorting procedure may be omitted for some or all of the DIMD-based modes, and they may be assigned to the front of the MPM list (e.g., the general MPM list or the first MPM list). In this case, the planner mode may be positioned at the front, and some or all of the DIMD-based modes may be assigned behind the planner mode. Alternatively, if the planner mode is not included in the MPM list, some or all of the DIMD-based modes may be assigned to the front. Alternatively, the DIMD-based modes may be compared with intra prediction modes derived from neighboring blocks, and modes that overlap each other may be assigned to the front of the MPM list since they have a high correlation with the current block. In other words, the pruning procedure and the sorting procedure may be combined. For example, according to the previous method, a pruning procedure and a sorting procedure must be performed respectively to remove duplication between intra prediction modes derived from surrounding blocks (first group) and DIMD-based modes (second group), but according to the present method, the sorting procedure can be omitted by assigning priorities to duplicated modes while performing a duplication check.

Figure 10 exemplarily shows DIMD-based modes and intra prediction modes derived from surrounding blocks.

Referring to Fig. 10, when mode B and mode C overlap between two groups, the modes B and C can be assigned priority in the MPM list. For example, in this case, mode B and mode C can be assigned to the very front of the MPM list or (immediately) behind the planner mode.

In this case, for example, the priority between mode B and mode C can be based on the priority between DIMD-based modes. In the case of DIMD-based modes, since the priority has already been derived based on HoG, the priority can be determined without going through a separate SAD-based sorting procedure.

Meanwhile, intra prediction modes used in coding previous blocks within a certain region based on history can be stored and reused. This can be called HIPM (history-based intra prediction mode). The HIPM can be called by various names such as HMPM (history-based MPM), history-based candidate mode, etc. The certain region can include, for example, a CTU, a CTU row, a plurality of CTU rows, a slice, a tile, etc. The plurality of CTU rows can include, for example, an n-th CTU row and an n-1-th CTU row where a current block is located. In this case, the HIPM list can perform reordering based on the frequency of the intra prediction mode. That is, an intra prediction mode with a high occurrence frequency within a certain region has a higher priority than an intra prediction mode with a low occurrence frequency, and can be allocated to the front of the list. When allocated to the front of the list, the mode can be indicated with a lower index value.

Figure 11 exemplarily shows intra prediction modes of previous blocks within a certain area.

Referring to Fig. 11, the HIPM list can be configured such that a specific mode (e.g. Mode #50) has the highest frequency and thus has a high priority. The list can be called a buffer. In this case, for example, Mode #28 can be assigned to the next position after Mode #50 since it has the next highest frequency.

In constructing the above HIPM list, the intra prediction mode of the previous block within the predetermined area may be unavailable. This may be, for example, when the previous block is coded based on inter prediction (inter coded). In this case, the intra prediction mode of the previous block may be considered as a planar mode or a DC mode. Alternatively, the intra prediction mode of the previous block may be omitted when constructing the HIPM list.

The above HIPM list can be used to construct the MPM list. For example, the MPM list can be constructed based on x candidates in order of priority among the candidates of the HIPM list. That is, the MPM list can include x candidates in order of priority among the candidates of the HIPM list. Alternatively, the candidates of the MPM list can be derived based on the intra prediction modes (first group) and DIMD-based modes (second group) derived from the surrounding blocks and/or the candidates of the HIPM list (third group). In this case, a pruning and sorting procedure can be performed, and only the preceding p can be extracted through the sorting procedure.

Meanwhile, as described above, there may be a general MPM list, a first MPM list, and/or a second MPM list. The second MPM list may be divided into four groups. In this case, for example, the PMPM flag may be signaled first, and when the PMPM flag is 1, the PMPM index may be signaled. Here, the PMPM flag may indicate whether the intra prediction mode of the current block exists in the first MPM list. For example, the GMPM flag may be signaled before the PMPM flag. In this case, the PMPM flag and the SMPM flag described below may be signaled when the GMPM flag is 1. When the PMPM flag is 0, the SMPM flag may be signaled first, and when the SMPM flag is 1, the group index (SMPM group index) for the second MPM list may be parsed/signaled first, and the mode index (SMPM mode index) may be signaled/parsed later. Information about intra prediction mode (e.g., GMPM flag, PMPM flag, PMPM index, SMPM flag, SMPM group index and/or SMPM mode index) can be signaled via CU syntax. When the GMPM flag is 0, the PMPM flag and SMPM flag can be derived as 0 without signaling/coding. The GMPM flag can also be simply called the MPM flag. The GMPM flag can be omitted in some cases.

Information about intra prediction can be signaled, for example, as follows. Information about intra prediction can include one or more syntax elements. The same applies hereinafter.

In this case, for example, smpm_group_idx can be fixed-length binarized, and smpm_mode_idx can be fixed-length binarized.

As another example, smpm_group_idx can be truncated rice (or truncated unary) binarized. In this case, smpm_mode_idx can be fixed-length binarized. Or, smpm_mode_idx can determine a binarization method depending on the smpm_group_idx value. For example, if the smpm_group_idx value is 0 or 1, it can be truncated rice (or truncated unary) binarized, and in other cases, it can be fixed-length binarized. smpm_mode_idx can be context model-based coding, and in this case, the context model of smpm_mode_idx can be determined differently based on the size of the current block and/or the smpm_group_idx value. For example, the context model can be directed based on the context index increment (ctxInc), and the ctxInc for a bin of smpm_mode_idx can be determined differently based on the smpm_group_idx value, for example, as follows. As described below, smpm_group_idx can be replaced with smpm_group_flag.

The second MPM list can be divided into two groups. In this case, the number of candidates in the first group can be the same as or different from the number of candidates in the second group. For example, the number of candidates in the second MPM list can be 16, 12, or 8. In this case, the first group can contain 4 or 8 candidates, and the second group can contain 8 candidates.

For example, smpm_group_flag can be used instead of smpm_group_idx. smpm_group_flag can indicate whether the intra prediction mode of the current block is included in the second candidate group among the candidate groups of the second MPM list. smpm_mode_idx can determine a binarization method according to the smpm_group_flag value. For example, if the smpm_group_flag value points to the first group (ex. value 1), truncated rice (or truncated unary) binarization is performed, and in other cases, fixed-length binarization can be performed.

In this case, for example, information about intra prediction can be signaled as follows:

Alternatively, smpm_idc may be used instead of smpm_flag and/or smpm_group_flag (or smpm_group_idx).

In this case, for example, information about intra prediction can be signaled as follows:

smpm_idc can indicate whether the intra prediction mode of the current block is included in the second MPM list and, if so, in which group it is included. smpm_idc can be binarized based on truncated rice (or truncated unary).

The following table shows an example of binarization of smpm_idc.

smpm_idc Description Binarization 0 Not included 0 1 In First group 10 2 In the second group 11

smpm_mode_idx can be called smpm_idx. smpm_idx can be binarized based on truncated rice (or truncated unary).

The following table shows an example of binarization of smpm_idx.

smpm_idx Description Binarization 0 First candidate 0 1 Second candidate 10 2 Third candidate 110 3 Fourth candidate 111

Alternatively, the binarization of smpm_idx can be based on smpm groups. For example, the binarization of smpm_idx can be based on smpm_idc value. For example, when smpm_idc value is 1, smpm_idx can be binarized based on truncated rice (or truncated unary), and when smpm_idc value is 2, smpm_idx can be binarized based on fixed length. In this case, for example, the number of candidates in the first group can be 4, and the number of candidates in the second group can be 8. Through this, the maximum number of bins for the binarization for the worst case can be matched.

smpm_idx (for first group) Description Binarization 0 First candidate 0 1 Second candidate 10 2 Third candidate 110 3 Fourth candidate 111

smpm_idx (for non-first group) Description Binarization 0 First candidate 000 1 Second candidate 001 2 Third candidate 010 3 Fourth candidate 011 4 Fifth candidate 100 5 Sixth candidate 101 6 Seventh candidate 110 7 Eighth candidate 111

As described above, for the first group of the second MPM list, since the correlation with the current block is high, variable-length binarization can be performed considering the best case, and for the second group of the second MPM list, since the correlation with the current block is relatively low, fixed-length binarization can be performed considering the worst case.

The above smpm_idx can be context-based coded (e.g. CABAC), in which case the context index (or context index increment) of bin 0 can be set differently depending on the case. The context model of smpm_idx can be determined differently based on the size of the current block and/or the smpm_idc value. For example, the context index (or context index increment) of bin 0 of smpm_idx can be set differently based on smpm_idc (or smpm_group_idx or smpm_group_flag).

For example, if the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is greater than 1, the context index increment of bin 0 of smpm_idx can be 1. If the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is 1, the context index increment of bin 0 of smpm_idx can be 0. Or vice versa. In the tables below, smpm_idc can be replaced with smpm_group_idx or smpm_group_flag. In this case, the value of smpm_group_idx or smpm_group_flag, which is the condition, can be set to 1 less than the value of smpm_idc.

Through context-based coding as described above, a context model can be adaptively allocated to information related to intra prediction modes, and entropy coding efficiency can be efficiently increased.

FIG. 12 schematically illustrates a video/image encoding method according to an embodiment(s) of the present disclosure. The method disclosed in FIG. 12 may be performed by the encoding device disclosed in FIG. 2. Specifically, for example, S1200 to S1220 of FIG. 12 may be performed by the prediction unit (220) of the encoding device (200), and S1230 of FIG. 12 may be performed by the entropy encoding unit (240) of the encoding device (200). The method disclosed in FIG. 12 may include the embodiments described above in the present disclosure.

Referring to FIG. 12, the encoding device derives an MPM list for the current block (S1200).

The above MPM list may include intra prediction mode candidates for the current block. The MPM list may include a first MPM list, a second MPM list, and a third MPM list. In this case, the second MPM list and the first MPM list may not overlap each other, and the second MPM list and the third MPM list may overlap each other. For example, candidates included in the second MPM list may not be included in the first MPM list, and candidates included in the second MPM list may be included in the third MPM list. The first MPM list may be called a PMPM list. The second MPM list may be called an SMPM list. The third MPM list may be called a GMPM list.

The above MPM list is derived based on a first intra prediction mode group and a second intra prediction mode group, wherein the first intra prediction mode group includes intra prediction modes of surrounding blocks of the current block, and the second intra prediction mode group includes DIMD (decoder side intra mode derivation) based intra prediction modes for the current block.

The intra prediction mode of the second intra prediction mode group may have a higher priority than the intra prediction mode of the first intra prediction mode group and may be assigned earlier in the MPM list.

An overlapping mode between the intra prediction modes of the first intra prediction mode group and the intra prediction modes of the second intra prediction mode group can be derived. In this case, the overlapping mode has a higher priority than the non-overlapping mode and can be allocated to the front of the MPM list.

Overlapping modes between the intra prediction modes of the first intra prediction mode group and the intra prediction modes of the second intra prediction mode group can be derived. When there are multiple overlapping modes, the priority between the overlapping modes can be based on the priority between the DIMD-based intra prediction modes.

The above MPM list can be further derived based on a third intra prediction mode group. The third intra prediction mode group can include history-based intra prediction modes within a history-based intra prediction mode (HIPM) buffer.

The HIPM buffer may include intra prediction modes of previously decoded blocks of a specific area where the current block is located and which are not adjacent to the current block. The intra prediction modes in the HIPM buffer may be reordered based on the occurrence frequency of the intra prediction modes in the specific area. The intra prediction mode of an unavailable block in the specific area may be set to be the same as a predetermined intra prediction mode or a DIMD-based intra prediction mode for the specific block. The intra prediction mode of a specific block coded based on inter prediction or IBC (intra block copy) prediction in the specific area may be set to be the same as a predetermined intra prediction mode or a DIMD-based intra prediction mode for the specific block. The predetermined intra prediction mode may be a planar mode or a DC mode. The HIPM buffer may be initialized in units of CTU rows. The HIPM buffer may also be initialized in units of CTU, multiple CTU rows, slices, or tiles.

The encoding device derives an intra prediction mode of the current block (S1210). The encoding device can derive an intra prediction mode to be applied to the current block from among intra prediction modes based on the RD cost.

The encoding device generates prediction-related information (S1220). The encoding device can generate the prediction-related information based on the derived intra prediction mode and the MPM list.

For example, the prediction related information may include at least one of a PMPM flag, a PMPM index, an SMPM flag, an SMPM group index, or an SMPM mode index. In this case, the PMPM flag indicates whether the intra prediction mode of the current block exists in the first MPM list, the PMPM index is an index for indicating a candidate within the first MPM list, the SMPM flag indicates whether the intra prediction mode of the current block exists in the second MPM list, the SMPM group index is an index for indicating an SMPM group within the second MPM list, and the SMPM mode index is an index for indicating a candidate within the SMPM group. The SMPM group index and the SMPM mode index may be signaled when the value of the SMPM flag is 1.

The binarization method of the above SMPM mode index can be determined differently based on the value of the above SMPM group index.

The above SMPM mode index is encoded based on a context model, and the context model of the SMPM mode index can be determined based on the value of the SMPM group index.

The above SMPM group index points to a first SMPM group or a second SMPM group, and the number of candidates of the first SMPM group may be different from the number of candidates of the second SMPM group. For example, the number of candidates of the second SMPM group may be twice the number of candidates of the first SMPM group.

As another example, the prediction related information may include at least one of a PMPM flag, a PMPM index, an SMPM flag, an SMPM group flag, or an SMPM mode index. The PMPM flag indicates whether the intra prediction mode of the current block exists in the first MPM list, the PMPM index is an index for indicating a candidate within the first MPM list, the SMPM flag indicates whether the intra prediction mode of the current block exists in the second MPM list, the SMPM group flag is an index for indicating an SMPM group within the second MPM list, and the SMPM mode index is an index for indicating a candidate within the SMPM group. The SMPM group flag and the SMPM mode index may be signaled when the value of the SMPM flag is 1. As described above, at least one of the SMPM flag and the SMPM group flag may be replaced with SMPM indication (smpm_idc) information.

The binarization method of the above SMPM mode index can be determined differently based on the value of the SMPM group flag. For example, based on the case where the value of the SMPM group flag is 1, the SMPM mode index can be binarized based on a truncated unary basis, and based on the case where the value of the SMPM group flag is 0, the SMPM mode index can be binarized based on a fixed length basis.

The above SMPM mode index is encoded based on a context model, and the context model of the above SMPM mode index can be determined based on the value of the SMPM group flag.

The encoding device encodes image information including prediction-related information (S1230). For example, the prediction-related information may include CU syntax for the current block. The image information may be referred to as video information.

Additionally, the image information may include various information according to an embodiment of the present disclosure. For example, the image information may include information disclosed in at least one of the tables described above.

Meanwhile, the image information may include residual information. The residual information is information about residual samples. The residual information may include information about quantized transform coefficients for the residual samples.

The encoded image information can be output in the form of a bitstream. The bitstream can be transmitted to a decoding device via a network or storage medium. For example, the image data including the bitstream can be transmitted to a decoding device via a transmission device (or transmission unit). In this case, the image data including the bitstream can be transmitted to the decoding device via a streaming server.

In addition, as described above, the encoding device can generate a restored picture (including restored samples and restored blocks) based on the reference samples and the residual samples. This is to derive the same prediction result as performed in the decoding device from the encoding device, and thereby increase coding efficiency. Accordingly, the encoding device can store the restored picture (or restored samples, restored blocks) in memory and utilize it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. can be further applied to the restored picture.

According to the above-described embodiment(s), it is possible to efficiently indicate the prediction mode applied to the current block, and reduce the amount of data of side information. In addition, it is possible to construct an MPM list by considering not only the correlation with the surrounding blocks but also the correlation with the surrounding restoration samples, and it is possible to increase the probability that the intra prediction mode of the current block is constructed as a top candidate of the MPM list, and it is possible to increase the signaling efficiency of the intra prediction mode.

FIG. 13 schematically illustrates a video/image decoding method according to an embodiment(s) of the present disclosure. The method disclosed in FIG. 13 may be performed by the decoding device disclosed in FIG. 3. Specifically, for example, S1300 of FIG. 13 may be performed by the entropy decoding unit (310) of the decoding device (300), and S1310 to S1330 may be performed by the prediction unit (330) of the decoding device (300). The method disclosed in FIG. 13 may include the embodiments described above in the present disclosure.

Referring to FIG. 13, the decoding device obtains prediction-related information through the bitstream (S1300). The decoding device can obtain image information including the prediction-related information through the bitstream. The image information may further include residual information as described above.

For example, the prediction related information may include at least one of a PMPM flag, a PMPM index, an SMPM flag, an SMPM group index, or an SMPM mode index. In this case, the PMPM flag indicates whether the intra prediction mode of the current block exists in the first MPM list, the PMPM index is an index for indicating a candidate within the first MPM list, the SMPM flag indicates whether the intra prediction mode of the current block exists in the second MPM list, the SMPM group index is an index for indicating an SMPM group within the second MPM list, and the SMPM mode index is an index for indicating a candidate within the SMPM group. The SMPM group index and the SMPM mode index may be signaled when the value of the SMPM flag is 1.

The binarization method of the above SMPM mode index can be determined differently based on the value of the above SMPM group index.

The above SMPM mode index is decoded based on a context model, and the context model of the above SMPM mode index can be determined based on the value of the above SMPM group index.

The above SMPM group index points to a first SMPM group or a second SMPM group, and the number of candidates of the first SMPM group may be different from the number of candidates of the second SMPM group. For example, the number of candidates of the second SMPM group may be twice the number of candidates of the first SMPM group.

As another example, the prediction related information may include at least one of a PMPM flag, a PMPM index, an SMPM flag, an SMPM group flag, or an SMPM mode index. The PMPM flag indicates whether the intra prediction mode of the current block exists in the first MPM list, the PMPM index is an index for indicating a candidate within the first MPM list, the SMPM flag indicates whether the intra prediction mode of the current block exists in the second MPM list, the SMPM group flag is an index for indicating an SMPM group within the second MPM list, and the SMPM mode index is an index for indicating a candidate within the SMPM group. The SMPM group flag and the SMPM mode index may be signaled when the value of the SMPM flag is 1. As described above, at least one of the SMPM flag and the SMPM group flag may be replaced with SMPM indication (smpm_idc) information.

The binarization method of the above SMPM mode index can be determined differently based on the value of the SMPM group flag. For example, based on the case where the value of the SMPM group flag is 1, the SMPM mode index can be binarized based on a truncated unary basis, and based on the case where the value of the SMPM group flag is 0, the SMPM mode index can be binarized based on a fixed length basis.

The above SMPM mode index is decoded based on a context model, and the context model of the above SMPM mode index can be determined based on the value of the SMPM group flag.

The decoding device derives the MPM list for the current block (S1310).

The above MPM list may include intra prediction mode candidates for the current block. The MPM list may include a first MPM list, a second MPM list, and a third MPM list. In this case, the second MPM list and the first MPM list may not overlap each other, and the second MPM list and the third MPM list may overlap each other. For example, candidates included in the second MPM list may not be included in the first MPM list, and candidates included in the second MPM list may be included in the third MPM list. The first MPM list may be called a PMPM list. The second MPM list may be called an SMPM list. The third MPM list may be called a GMPM list.

The above MPM list is derived based on a first intra prediction mode group and a second intra prediction mode group, wherein the first intra prediction mode group includes intra prediction modes of surrounding blocks of the current block, and the second intra prediction mode group includes DIMD (decoder side intra mode derivation) based intra prediction modes for the current block.

The intra prediction mode of the second intra prediction mode group may have a higher priority than the intra prediction mode of the first intra prediction mode group and may be assigned earlier in the MPM list.

An overlapping mode between the intra prediction modes of the first intra prediction mode group and the intra prediction modes of the second intra prediction mode group can be derived. In this case, the overlapping mode has a higher priority than the non-overlapping mode and can be allocated to the front of the MPM list.

Overlapping modes between the intra prediction modes of the first intra prediction mode group and the intra prediction modes of the second intra prediction mode group can be derived. When there are multiple overlapping modes, the priority between the overlapping modes can be based on the priority between the DIMD-based intra prediction modes.

The above MPM list can be further derived based on a third intra prediction mode group. The third intra prediction mode group can include history-based intra prediction modes within a history-based intra prediction mode (HIPM) buffer.

The HIPM buffer may include intra prediction modes of previously decoded blocks of a specific area where the current block is located and which are not adjacent to the current block. The intra prediction modes in the HIPM buffer may be reordered based on the occurrence frequency of the intra prediction modes in the specific area. The intra prediction mode of an unavailable block in the specific area may be set to be the same as a predetermined intra prediction mode or a DIMD-based intra prediction mode for the specific block. The intra prediction mode of a specific block coded based on inter prediction or IBC (intra block copy) prediction in the specific area may be set to be the same as a predetermined intra prediction mode or a DIMD-based intra prediction mode for the specific block. The predetermined intra prediction mode may be a planar mode or a DC mode. The HIPM buffer may be initialized in units of CTU rows. The HIPM buffer may also be initialized in units of CTU, multiple CTU rows, slices, or tiles.

The decoding device derives the intra prediction mode of the current block (S1320). The decoding device can derive the intra prediction mode of the current block based on the MPM list and the prediction-related information.

The decoding device generates prediction samples based on the derived intra prediction mode (S1330). For example, the decoding device may generate the prediction samples for the current block based on the derived intra prediction modes and surrounding reference samples. In this case, as described above, a prediction sample filtering procedure may be further performed on all or part of the prediction samples of the current block, depending on the case.

The decoding device can generate restoration samples based on prediction samples of the current block. For example, the decoding device can generate restoration samples for the current block based on residual samples for the current block and the prediction samples. The residual samples for the current block can be generated based on received residual information. In addition, the decoding device can generate a restoration picture including the restoration samples, for example. As described above, an in-loop filtering procedure, etc. can be further applied to the restoration picture.

According to the above-described embodiment(s), it is possible to efficiently indicate the prediction mode applied to the current block, and reduce the amount of data of side information. In addition, it is possible to construct an MPM list by considering not only the correlation with the surrounding blocks but also the correlation with the surrounding restoration samples, and it is possible to increase the probability that the intra prediction mode of the current block is constructed as a top candidate of the MPM list, and it is possible to increase the signaling efficiency of the intra prediction mode.

In the above-described embodiments, the methods are described based on a flow chart as a series of steps or blocks, but the embodiments are not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will appreciate that the steps depicted in the flow chart are not exclusive, and other steps may be included or one or more of the steps in the flow chart may be deleted without affecting the scope of the embodiments of the present disclosure.

The method according to the embodiments of the present disclosure described above may be implemented in the form of software, and the encoding device and/or the decoding device according to the present disclosure may be included in a device that performs image processing, such as a TV, a computer, a smartphone, a set-top box, a display device, etc.

The above-described embodiments of the present disclosure may also be implemented in the form of a recording medium containing computer-executable (program) instructions, such as program modules, executed by a computer. The modules may be stored in a memory and executed by a processor. The memory may be internal or external to the processor and may be connected to the processor by various means known in the art. The computer-readable medium may be any available medium that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. The computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. The communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism, and includes any information delivery media.

In addition, the above-described embodiments of the present disclosure may be implemented as a computer program (or a computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor, and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language. In addition, the computer program may be recorded in a tangible computer-readable recording medium (e.g., a memory, a hard disk, a magnetic/optical medium, or a solid-state drive (SSD)).

Accordingly, the above-described embodiment of the present disclosure can be implemented by executing the computer program as described above by a computing device. The computing device can include at least some of a processor, a memory, a storage device, a high-speed interface connecting the memory and a high-speed expansion port, and a low-speed interface connecting the low-speed bus and the storage device. Each of these components is connected to each other using various buses, and can be mounted on a common motherboard or mounted in another suitable manner.

Here, the processor may process instructions within the computing device, such as instructions stored in a memory or storage device for displaying graphical information for providing a GUI (Graphical User Interface) on an external input/output device, such as a display connected to a high-speed interface. In other embodiments, multiple processors and/or multiple buses may be utilized, as appropriate, together with multiple memories and memory types. Additionally, the processor may be implemented as a chipset comprising multiple independent analog and/or digital processors.

Memory also stores information within a computing device. For example, memory may be comprised of volatile memory units or a collection thereof. For another example, memory may be comprised of nonvolatile memory units or a collection thereof. Memory may also be another form of computer-readable media, such as, for example, a magnetic or optical disk.

And the storage device can provide a large amount of storage space to the computing device. The storage device can be a computer-readable medium or a configuration including such a medium, and can include, for example, devices within a storage area network (SAN) or other configurations, and can be a floppy disk device, a hard disk device, an optical disk device, a tape device, flash memory, or other similar semiconductor memory device or array of devices.

Additionally, the network can be implemented as a wired network such as a Local Area Network (LAN), a Wide Area Network (WAN), or a Value Added Network (VAN), or as various types of wireless networks such as a mobile radio communication network or a satellite communication network.

The present disclosure discussed above has been described with reference to the embodiments illustrated in the drawings, but this is merely exemplary, and those skilled in the art will understand that various modifications and variations of the embodiments are possible from this. That is, the scope of the present disclosure is not limited to the above-described implementation examples, and various modifications and improvements made by those skilled in the art using the basic concept of the implementation examples defined in the following claims also fall within the scope of the implementation examples. Therefore, the true technical protection scope of the present disclosure should be determined by the technical idea of the appended claims.

Claims (20)

In a video decoding method performed by a decoding device, A step of obtaining prediction-related information through a bitstream; A step for deriving a list of most probable modes (MPMs) for the current block; A step of deriving an intra prediction mode of the current block based on the above MPM list and the above prediction-related information; and A step of generating prediction samples for the current block based on the derived intra prediction mode is included. A video decoding method, characterized in that the MPM list includes a first MPM list, a second MPM list, and a third MPM list, and candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list. In the first paragraph, The above MPM list is derived based on the first intra prediction mode group and the second intra prediction mode group, The above first intra prediction mode group includes intra prediction modes of surrounding blocks of the current block, A video decoding method, characterized in that the second intra prediction mode group includes DIMD (decoder side intra mode derivation) based intra prediction modes for the current block. In the second paragraph A video decoding method, characterized in that the intra prediction mode of the second intra prediction mode group has a higher priority than the intra prediction mode of the first intra prediction mode group and is allocated to a higher position in the MPM list. In the second paragraph Further comprising a step of deriving an overlapping mode between the intra prediction modes of the first intra prediction mode group and the intra prediction modes of the second intra prediction mode group, A video decoding method, characterized in that the overlapping mode has a higher priority than the non-overlapping mode and is allocated to a higher position in the MPM list. In the second paragraph, Further comprising a step of deriving overlapping modes between intra prediction modes of the first intra prediction mode group and intra prediction modes of the second intra prediction mode group, An image decoding method, characterized in that when there are multiple overlapping modes, the priority between the overlapping modes is based on the priority between the DIMD-based intra prediction modes. In the first paragraph, The above MPM list is derived further based on the third intra prediction mode group, A video decoding method, characterized in that the third intra prediction mode group includes history-based intra prediction modes within a HIPM (history-based intra prediction mode) buffer. In Article 6, A video decoding method, characterized in that the HIPM buffer includes intra prediction modes of blocks that were previously decoded in a specific area where the current block is located and are not adjacent to the current block. In Article 7, An image decoding method, characterized in that the intra prediction modes within the HIPM buffer are reordered based on the occurrence frequency of the intra prediction mode within the specific region. In Article 7, An image decoding method, characterized in that the intra prediction mode of a specific block coded based on inter prediction or IBC (intra block copy) prediction within the specific region is set to be the same as a predetermined intra prediction mode or a DIMD-based intra prediction mode for the specific block. In Article 6 A video decoding method, characterized in that the above HIPM buffer is initialized in units of CTU rows. In the first paragraph, The above prediction related information includes at least one of a PMPM flag, a PMPM index, an SMPM flag, an SMPM group index, or an SMPM mode index, The above PMPM flag indicates whether the intra prediction mode of the current block exists in the first MPM list, The above PMPM index is an index for indicating a candidate within the first MPM list, The above SMPM flag indicates whether the intra prediction mode of the current block exists in the second MPM list, The above SMPM group index is an index for indicating an SMPM group within the second MPM list, The above SMPM mode index is an index for indicating a candidate within the above SMPM group. A video decoding method, characterized in that the SMPM group index and the SMPM mode index are signaled when the value of the SMPM flag is 1. In Article 11 An image decoding method, characterized in that the binarization method of the above SMPM mode index is determined differently based on the value of the above SMPM group index. In Article 11 The above SMPM mode index is decoded based on the context model, An image decoding method, characterized in that the context model of the above SMPM mode index is determined based on the value of the above SMPM group index. In Article 11 The above SMPM group index points to the first SMPM group or the second SMPM group, A video decoding method, characterized in that the number of candidates of the first SMPM group is different from the number of candidates of the second SMPM group. In the first paragraph, The above prediction related information includes at least one of a PMPM flag, a PMPM index, an SMPM flag, an SMPM group flag or an SMPM mode index, The above PMPM flag indicates whether the intra prediction mode of the current block exists in the first MPM list, The above PMPM index is an index for indicating a candidate within the first MPM list, The above SMPM flag indicates whether the intra prediction mode of the current block exists in the second MPM list, The above SMPM group flag is an index for indicating an SMPM group within the second MPM list, The above SMPM mode index is an index for indicating a candidate within the above SMPM group. A video decoding method, characterized in that the SMPM group flag and the SMPM mode index are signaled when the value of the SMPM flag is 1. In Article 15 An image decoding method, characterized in that the binarization method of the above SMPM mode index is determined differently based on the value of the above SMPM group flag. In Article 16, Based on the case where the value of the above SMPM group flag is 1, the above SMPM mode index is binarized based on truncated unitary, A video decoding method, characterized in that the SMPM mode index is binarized based on a fixed length when the value of the above SMPM group flag is 0. In Article 15 The above SMPM mode index is decoded based on the context model, A video decoding method, characterized in that the context model of the above SMPM mode index is determined based on the value of the above SMPM group flag. In a video encoding method performed by an encoding device, A step for deriving a list of most probable modes (MPMs) for the current block; and A step of deriving an intra prediction mode of the current block; A step of generating prediction-related information for indicating an intra prediction mode of the current block based on the above MPM list; and Comprising a step of encoding image information including the above prediction-related information, A video encoding method, characterized in that the MPM list includes a first MPM list, a second MPM list, and a third MPM list, and candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list. In a transmission method for image data, Obtaining a bitstream generated by an image encoding method, wherein the image encoding method comprises the steps of: deriving a most probable mode (MPM) list for a current block; deriving an intra prediction mode of the current block; generating prediction-related information for indicating the intra prediction mode of the current block based on the MPM list; and encoding image information including the prediction-related information; and Comprising a step of transmitting image data including the above bitstream, A transmission method, characterized in that the MPM list includes a first MPM list, a second MPM list, and a third MPM list, and candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list.