HK1243257B - Apparatus for decoding and method for decoding, apparatus for encoding and method for encoding, method for storing a picture and method for transmitting a picture - Google Patents

Description

Decoding device and method, encoding device and method, and image storage and transmission method

This application is a divisional application, the application number of the parent application is 201180064204.6, the application date is November 4, 2011, and the name of the invention is “Image coding supporting block merging and skip mode”.

Technical Field

The present application relates to image and/or video coding, and in particular to a codec supporting block separation and skip mode.

Background Art

Many picture and/or video codecs process images in units of blocks. For example, predictive codecs use a block granularity to achieve a good compromise between, on the one hand, spending too much side information on prediction parameters, setting the prediction parameters very precisely at high spatial resolutions, and, on the other hand, setting the prediction parameters too coarsely, which increases the number of bits required to encode the prediction residual due to lower spatial resolution prediction parameters. In practice, the optimal settings for these prediction parameters lie somewhere between these two extremes.

Many attempts have been made to optimize the above-mentioned problem. For example, instead of using regular sub-segmentation of an image into blocks regularly arranged into columns and rows, multi-tree partitioning sub-segmentation seeks to increase the degree of freedom in sub-segmenting an image into blocks, while maintaining reasonable requirements for sub-segmentation information. However, even when using multi-tree sub-segmentation, which requires a considerable amount of data, the degree of freedom in sub-segmenting an image is quite limited.

In order to achieve a better compromise between signaling the amount of side information required for picture subdivisions on the one hand and signaling the degrees of freedom of the subdivided pictures on the other hand, block merging can be used to increase the number of possible picture subdivisions with a reasonable amount of additional data required to signal the merging information. For the merged blocks, the coding parameters need to be sent only once in the bitstream, similar to the case when the resulting group of merged blocks is a direct subdivision of the picture.

To further increase the efficiency of encoding image content, skip modes have been introduced in some block-based image codecs. These skip modes allow the encoder to suppress the transmission of residual data for certain blocks to the decoder. In other words, skip mode makes it possible to suppress the transmission of residual data for certain blocks. The ability to suppress the transmission of residual data for certain blocks allows for a wider range of granularity for encoding coding/prediction parameters, where an optimal trade-off between coding quality on the one hand and the overall bit rate consumed on the other can be expected: while increasing the spatial resolution of the coding/prediction parameters leads to an increase in the information rate, it also reduces the amount of residual data, thus lowering the bit rate at which the residual data must be encoded. However, the availability of skip modes allows for only a modest further increase in the granularity at which coding/prediction parameters are transmitted, while the residual data is so small that the corresponding transmission of the residual data can be omitted, resulting in unexpected coding rate savings.

However, due to the newly introduced redundancy caused by block merging and skip mode usage, there is still a need to achieve better coding performance.

It is therefore an object of the present invention to provide a coding concept with increased coding efficiency.

Summary of the Invention

The present invention contemplates that further increases in coding efficiency can be achieved if common signaling of merge and skip mode activation is used within the bitstream. Specifically, one of the possible states of one or more syntax elements within the bitstream can signal that, for a current sample set of a picture, the corresponding sample set is to be merged and encoded without a prediction residual and inserted into the bitstream. Furthermore, a common flag can jointly signal whether coding parameters associated with the current sample set are to be set based on a merge candidate or obtained from the bitstream, and whether the current sample set of the picture is to be reconstructed solely based on a prediction signal that depends on the coding parameters associated with the current sample set without any residual data, or is to be reconstructed using residual data within the bitstream by refining a prediction signal that depends on the coding parameters associated with the current sample set.

The inventors of the present invention have discovered that the introduction of joint signaling for merging activation on the one hand and skip mode activation on the other hand saves bit rate, since the additional overhead for signaling separate merging and/or skip mode activation can be reduced or may only have to be incurred if merging and skip mode are not activated at the same time.

Preferred embodiments of the present application will be described in more detail below with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 shows a block diagram of an apparatus for encoding according to an embodiment;

FIG2 shows a block diagram of an apparatus for encoding according to a more detailed embodiment;

FIG3 shows a block diagram of an apparatus for decoding according to an embodiment;

FIG4 shows a block diagram of an apparatus for decoding according to a more detailed embodiment;

FIG5 is a block diagram showing a possible internal structure of the encoder of FIG1 or FIG2;

FIG6 is a block diagram showing a possible internal structure of the decoder of FIG3 or FIG4;

FIG7A illustrates a possible sub-division of a picture into tree-root blocks, coding units (blocks), and prediction units (partitions);

FIG7B shows a sub-partition tree of the root block shown in FIG7A down to the partition level according to one illustrative example;

FIG8 shows an example of a set of possible supported separation patterns according to an embodiment;

FIG9 shows possible partitioning patterns that would be effectively generated from combining blocks and partitioning blocks when using block partitioning according to FIG8 ;

FIG10 illustrates, in exploded form, candidate blocks for a SKIP/DIRECT mode according to an embodiment;

11 to 13A and 13B illustrate a grammar portion of a grammar according to an embodiment; and

FIG. 14 illustrates exploded views of the definition of adjacent partitions for a partition according to one embodiment.

DETAILED DESCRIPTION

Regarding the following description, it should be noted that whenever the same reference symbols are used in different figures, the description of the corresponding components relative to one of these figures will apply equally to the other figures, as long as the description transferred from one figure to another does not conflict with the remaining description of the other figure.

Figure 1 shows an apparatus 10 for encoding a picture 20 into a bitstream 30. Of course, the picture 20 may be part of a video, in which case the encoder 10 will be a video encoder.

Although not explicitly shown in FIG1 , image 20 is represented as a sample array. The sample array of image 20 is divided into sample sets 40, which can be any sample set, for example, a sample set covering a non-overlapping, single-connected region of image 20. For ease of understanding, sample sets 40 are shown as, and are referred to below as, blocks 40, but the following description should not be construed as limiting to any particular type of sample set 40. According to one embodiment, sample sets 40 are rectangular and/or square blocks.

For example, the image 20 may be subdivided into a regular arrangement of blocks 40, whereby the blocks 40 are exemplarily arranged in columns and rows as shown in FIG1 . However, any other subdivision of the image 20 into blocks 40 is also possible. In particular, the subdivision of the image 20 into blocks 40 may be fixed, i.e. a predetermined value known to the decoder or signaled to the decoder within the bitstream 30. In particular, the blocks 40 of the image 20 may have a varying granularity. For example, a multi-tree subdivision, e.g. a quarter-tree subdivision, may be applied to the image 20 or to a regular pre-subdivision of the image 20 into regularly arranged root blocks in order to obtain blocks 40, in which case the blocks 40 form leaf blocks of the multi-tree subdivision of the root blocks.

In any case, the encoder 10 is configured to encode, for the current sample set 40, a flag into the bitstream 30 that collectively signals whether the coding parameters associated with the current sample set 40 are set according to a merge candidate or are taken from the bitstream 30, and whether the current sample set of the picture 20 is to be reconstructed solely based on a prediction signal that depends on the coding parameters associated with the current sample set, without any residual data, or is to be reconstructed by refining the prediction signal that depends on the coding parameters associated with the current sample set 40 using residual data within the bitstream 30. For example, encoder 10 is configured to encode a flag for the current sample set 40 and collectively signal it into bitstream 30 if, assuming a first state, the coding parameters associated with the current sample set 40 are set based on a merge candidate rather than being obtained from bitstream 30, and the current sample set of picture 20 is reconstructed solely based on a prediction signal that depends on the coding parameters associated with the current sample set, without any residual data, and if assuming any other state, the coding parameters associated with the current sample set 40 are obtained from bitstream 30, or the current sample set of picture 20 is reconstructed by refining the prediction signal that depends on the coding parameters associated with the current sample set 40 using residual data within bitstream 30. This means the following: Encoder 10 supports merging of blocks 40. This merging is authorized. That is, not every block 40 is merged. For some blocks 40, for example, it is appropriate to merge the current block 40 with a merge candidate, in terms of rate/distortion optimization, but the opposite is true for others. To determine whether a block 40 should be merged, encoder 10 determines a set or list of merge candidates and, for each of these merge candidates, whether merging the current block 40 with the merge candidate would be the best encoding choice, e.g., in terms of bitrate/distortion optimization. Encoder 10 is configured to determine the set or list of merge candidates for a current block 40 based on a previously encoded portion of bitstream 30. For example, encoder 10 obtains at least a portion of the set or list of merge candidates by receiving coding parameters associated with regionally and/or temporally neighboring blocks 40 that have been previously encoded, depending on the coding order applied by encoder 10. Temporal proximity means, for example, that a block of a previously encoded video picture, to which picture 20 belongs, has temporally neighboring blocks that are spatially positioned so as to spatially overlap the current block 40 of the current picture 20. Thus, for this portion of the set or list of merge candidates, there is a one-to-one association between each merge candidate and its spatially and/or temporally neighboring blocks. Each merge candidate has coding parameters associated with it. If the current block 40 is merged with any of the merge candidates, the encoder 10 sets the coding parameters of the current block 40 according to the merge candidate. For example, the encoder 10 may set the coding parameters of the current block 40 equal to those of the corresponding merge candidate, i.e., the encoder 10 may copy the coding parameters of the current block 40 from the corresponding merge candidate. Thus, for this description of the set or list of merge candidates, the coding parameters of a merge candidate are either directly adopted from a spatially and/or temporally neighboring block, or are derived from the coding parameters of the corresponding merge candidate by adopting the same, i.e., setting the merge candidate identically, but taking into account regional changes, e.g., by rescaling the adopted coding parameters according to the regional changes. For example, the coding parameters of at least a portion of the merged state may include motion parameters. However, the motion parameters may indicate different reference picture indices. More specifically, the adopted motion parameter may indicate a time interval between the current image and the reference image, and in merging the current block with a corresponding merge candidate having the corresponding motion parameter, the encoder 10 may be configured to adjust the motion parameter scale of the corresponding merge candidate so as to adapt its time interval to the time interval selected for the current block.

In any case, the merge candidates described so far share their associated coding parameters, and there is a one-to-one correlation between these merge candidates and neighboring blocks. Thus, merging blocks 40 with the merge candidates just described can be viewed as merging these blocks into one or more groups of blocks 40, whereby coding parameters do not vary across image 20 within these groups of blocks 40, except for scale adaptation or the like. In practice, merging with any of the merge candidates just described will reduce the scale at which coding parameters vary across image 20. Furthermore, merging with any of the merge candidates just described will result in additional degrees of freedom in subdividing image 20 into blocks 40 and groups of blocks 40, respectively. Thus, in this regard, merging blocks 40 into these groups of blocks can be viewed as causing encoder 10 to encode image 20 using coding parameters that vary across image 20 for these groups of blocks 40.

In addition to the merge candidates just mentioned, the encoder 10 may also add merge candidates to the set/list of merge candidates, which are the combination results of the coding parameters of two or more neighboring blocks, for example, their arithmetic mean, geometric mean or median of the coding parameters of the neighboring blocks and the like.

Thus, the encoder 10 effectively reduces the granularity at which coding parameters are explicitly transmitted within the bitstream 30, compared to the granularity defined by the subdivision of the image 20 into blocks 40. Some of these blocks 40 form block groups by using the merge selection outlined above to use one and its coding parameters. Some blocks are coupled to each other via merging, but use different coding parameters that are associated with each other via corresponding scaling and/or combining functions. Some blocks 40 are not subject to merging, and therefore the encoder 10 encodes these coding parameters directly into the bitstream 30.

The encoder 10 uses the coding parameters of the block 40 thus defined in order to determine a prediction signal for the image 20 . The encoder 10 determines the prediction signal block-wise, wherein the prediction signal depends on the coding parameters associated with the respective block 40 .

Another decision made by the encoder 10 is whether a residual, i.e., the difference between the prediction signal and the original image content of the corresponding local area of the current block 40, is to be sent in the bitstream 30. That is, the encoder 10 decides whether a skip mode is to be applied to the corresponding block 40. If the skip mode is applied, the encoder 10 encodes the image 20 within the current partial block 40 only in the form of a prediction signal derived from or dependent on the coding parameters associated with the corresponding block 40, and in the event that the skip mode is deactivated, the encoder 10 encodes the image 20 within the block 40 into the bitstream 30 using the prediction signal and the residual data.

To save bitrate for signaling the decision to merge, on the one hand, and skip mode, on the other, encoder 10 uses a single flag to signal both decisions for block 40. More precisely, the signaling can be implemented jointly so that activation of both merge and skip modes is indicated jointly within bitstream 30 by a flag for the corresponding block 40 assuming a first possible flag state. However, the other flag states of the flag only indicate to the decoder that either merge or skip mode is not activated. For example, encoder 10 may decide to activate merge for a particular block 40 but deactivate skip mode. In this case, encoder 10 uses the other flag state to signal deactivation of at least one of merge and skip modes within bitstream 30, while signaling activation of merge within bitstream 30 sequentially, for example, using another flag. Thus, encoder 10 only needs to send this further flag if merge and skip modes are not simultaneously activated for a block 40. In the embodiments described further below, the first flag is referred to as mrg_cbf or skip_flag, and the auxiliary merge indication flag is referred to as mrg or merge_flag. The inventors have discovered that the common use of this signaling state to jointly signal the activation of merge and skip modes can reduce the overall bit rate of the bitstream 30.

Regarding the signaling state just mentioned, it should be noted that this signaling state can be determined using the state of a bit in the bitstream 30. However, the encoder 10 can be configured to encode the bitstream 30 using entropy coding techniques, and thus the correspondence between the signaling state and the bitstream 30 may be more complex. In this case, the state can correspond to a bit in the bitstream 30 in the entropy decoding domain. Furthermore, the signaling state can correspond to one of two states specified for a codeword according to a variable length coding mechanism. In the case of arithmetic coding, the signaling state that commonly signals the activation of merge and skip modes can correspond to one of the symbols in the symbol literal of the arithmetic coding mechanism.

As described above, encoder 10 uses a flag within bitstream 30 to signal the simultaneous activation of merge and skip modes. As will be described in more detail below, this flag can be sent within a syntax element with more than two possible states. This syntax element can also, for example, signal other coding options. The details will be explained in more detail below. However, in that case, one of the possible states of one or more syntax elements will signal the simultaneous activation. That is, whenever the syntax element of the current block 40 just described determines this predetermined possible state, encoder 10 signals the activation of both merge and skip modes. The decoder therefore does not need to further signal the activation of merge and skip modes accordingly.

With respect to the above description, it should be noted that partitioning picture 20 into blocks 40 may not represent the optimal resolution for determining coding parameters for picture 20. Instead, encoder 10 may append further partitioning information to each block 40 to signal within bitstream 30 one of the supported partitioning patterns for partitioning the current block 40 into subblocks 50 and 60, i.e., sample subsets. In this case, simultaneous merge/skip mode decisions are made by encoder 10 on a per-block basis, so that coding parameters and, for example, separate auxiliary merge and/or skip mode decisions, are defined for picture 20 on a per-block basis, i.e., on subblocks 50 and 60 in the example shown for block 40 in FIG. Of course, a non-partitioning mode may represent one of the supported partitioning patterns, resulting in encoder 10 determining only one set of coding parameters for block 40. Regardless of the number of subblocks 50 and 60 in the corresponding separation pattern, the merge decision can apply to all subblocks, i.e., one or more subblocks. That is, if merge is activated for block 40, this activation can be effective for all subblocks. According to an embodiment described further below, the aforementioned common state of jointly signaling merge and skip mode activation can also simultaneously signal a non-separation pattern between the supported separation patterns for the current block 40, so that in the event that a flag or syntax element assumes this state, no further separation information for the current block needs to be sent. Of course, in addition to the activation of merge and skip mode, any other separation pattern between the supported separation patterns can also be indicated simultaneously.

According to some embodiments of the present application, encoder 10 avoids the bit efficiency loss resulting from the common use of block partitioning of block 40 on the one hand and from the merging of sub-blocks 50 and 60 on the other hand. To be more precise, for example, in the sense of bit rate/distortion optimization, encoder 10 can determine whether it is preferable to further partition block 40 and which supporting partitioning pattern should be used in the current block 40 in order to adapt the granularity at which certain coding parameters are set or defined within the current block 40 of picture 20. As will be described in more detail below, the coding parameters can, for example, represent prediction parameters, such as inter-frame prediction parameters. These inter-frame prediction parameters can, for example, include a reference picture index, a motion vector, and the like. Supported partitioning patterns may, for example, include a non-partitioning mode, i.e., a selection according to which the current block 40 is not further partitioned, a horizontal partitioning mode, i.e., a selection according to which the current block 40 is subdivided along a horizontally extending line into an upper or top portion and a lower or bottom portion, and a vertical partitioning mode, i.e., a selection according to which the current block 40 is vertically subdivided along a vertically extending line into a left portion and a right portion. Furthermore, supported partitioning patterns may also include a selection according to which the current block 40 is further regularly subdivided into four further blocks, each assuming a fourth of the current block 40. Furthermore, the partitioning may involve all blocks 40 of the image 20 or only a suitable subset thereof, e.g., those having a certain coding mode associated therewith, such as inter-frame prediction mode. Similarly, it should be noted that merging, as such, may only be applicable to certain blocks, e.g., those coded in inter-frame prediction mode. According to an embodiment further described below, the states described above also signal that the corresponding block is in inter-prediction mode rather than intra-prediction mode. Thus, a state of the above flag for block 40 signals that the block is not further partitioned and that merging and skip modes are activated for the inter-prediction coded block. However, as an auxiliary decision in the case where the flag assumes another state, each partition or sample subset 50 and 60 may be accompanied by a further flag within the bitstream 30 to signal whether merging is to be applied to the corresponding partition 50 and 60. Furthermore, different subsets of supported partition modes may be available for block 40, for example, in the case of multi-tree sub-partitioned leaf blocks, in combination or depending on the block size, sub-partition level, or the like of block 40.

That is, the subdivision of image 20 into blocks, in particular to obtain blocks 40, can be fixed or signaled within the bitstream. Similarly, the partitioning pattern to be used to further partition the current block 40 can be signaled within bitstream 30 in the form of partitioning information. Therefore, the partitioning information can be considered an extension of the subdivision of image 20 into blocks 40. On the other hand, the original granularity of the subdivision of image 20 into blocks 40 can still be maintained. For example, encoder 10 can be configured to signal within bitstream 30 the coding mode to be used for the corresponding portion of image 20 or block 40 using the granularity defined by block 40, and encoder 10 can be configured to vary the coding parameters of the corresponding coding mode within the corresponding block 40 by increasing the granularity defined by the corresponding partitioning pattern selected for the corresponding block 40. For example, the coding mode signaled at the granularity of the block 40 may distinguish between an intra prediction mode, an inter prediction mode, and the like, such as a temporal inter prediction mode, an inter-view prediction mode, etc. The type of coding parameters associated with one or more sub-blocks (partitions) resulting from the partitioning of the respective block 40 then depends on the coding mode assigned to the respective block 40. For example, for an intra-coded block 40, the coding parameters may include a spatial direction along which the image content of a previously decoded portion of the image 20 is used to fill the respective block 40. In the case of an inter-coded block 40, the coding parameters may include, in particular, a motion vector for motion-compensated prediction.

FIG1 exemplarily illustrates a current block 40 that has been subdivided into two subblocks 50 and 60. In particular, a vertical partitioning pattern is illustrated. The smaller blocks 50 and 60 may also be referred to as subblocks 50 and 60, or partitions 50 and 60, or prediction units 50 and 60. In particular, encoder 10 may be configured to, when one of the signaled supported partitioning patterns specifies a subdivision of the current block 40 into two or more further blocks 50 and 60, remove, for all further blocks except a first subblock of subblocks 50 and 60 in coding order, from the set of coding parameter candidates for the respective subblock, coding parameter candidates having coding parameters identical to coding parameters associated with any subblock that, when merged with the respective subblock, would form one of the supported partitioning patterns. To be more precise, for each supported partitioning pattern, a coding order is defined within the generated one or more partitioning patterns 50 and 60. In the case of FIG1 , the coding order is exemplarily shown by arrow 70, which defines that the left partition 50 is coded before the right partition 60. In the case of a horizontal partition mode, it can be defined that the upper partition is coded before the lower partition. In any case, the encoder 10 is configured to remove, for the second partition 60 in the coding order 70, from the set of coding parameter candidates for the corresponding second partition 60, the coding parameter candidates having the same coding parameters as the coding parameters associated with the first partition 50, so as to avoid the occurrence of such a merge, that is, in fact, the partitions 50 and 60 will both have the same coding parameters associated with , which can actually be equivalently produced at a lower coding rate by selecting a non-partition mode for the current block 40.

For greater precision, encoder 10 can be configured to efficiently use block merging with block partitioning. Regarding block merging, encoder 10 can determine a corresponding set of coding parameter candidates for each of partitions 50 and 60. The encoder can be configured to determine the set of coding parameter candidates for each of partitions 50 and 60 based on the coding parameters of previously decoded blocks. In particular, at least some of the coding parameter candidates within the set of coding parameter candidates can be equal to, i.e., can be adopted from, the coding parameters of a previously decoded partition. Alternatively, at least some of the coding parameter candidates can be derived from coding parameter candidates associated with more than one previously encoded partition via a suitable combination (e.g., median, mean, or the like). However, because the encoder 10 is configured to make a decision on the reduced set of coding parameter candidates and, if more than one coding parameter candidate remains after the removal, for each non-first partition 60, to select between the remaining non-removed coding parameter candidates so as to set the coding parameters associated with the respective partition depending on a non-removed or selected coding parameter candidate, the encoder 10 is configured to perform the removal so that the coding parameter candidates that would effectively result in the re-association of the partitions 50 and 60 are removed. That is, syntactic clusters are effectively avoided, according to which encoding of a valid partition situation would be more complex than directly signaling the situation of this partition by using partition information alone.

Furthermore, since the set of coding parameter candidates is smaller, the amount of side information required to encode merge information into the bitstream 30 can be reduced due to the lower number of elements in these candidate sets. In particular, since the decoder can determine and sequentially reduce the set of coding parameter candidates in the same manner as the encoder of FIG. 1 , the encoder 10 of FIG. 1 can utilize the reduced set of coding parameter candidates (e.g., by using fewer bits) to insert a syntax element into the bitstream 30 indicating which of the non-removed coding parameter candidates is to be used for merging. Of course, in the case where the number of non-removed coding parameter candidates for the corresponding partition is only one, the introduction of a syntax element into the bitstream 30 can be completely suppressed. In any case, since the merging, i.e., setting the coding parameters associated with the corresponding partition based on one of the remaining, or selected, non-removed coding parameter candidates, the encoder 10 can completely suppress the insertion of a new coding parameter for the corresponding partition into the bitstream 30, thereby also reducing side information. According to some embodiments of the present application, the encoder 10 may be configured to signal within the bitstream 30 refined information for refining the remaining one or the selected one of the corresponding separated coding parameter candidates.

Based on the possibilities just described for reducing the list of merging candidates, encoder 10 can be configured to determine the merging candidates to be removed by comparing their coding parameters with the coding parameters of partitions whose merging would produce another supported partition pattern. This way of processing the coding parameter candidates will effectively remove at least one coding parameter candidate in the scenario shown in FIG. 1 , for example, assuming that the coding parameters of left partition 50 form an element of the set of coding parameter candidates for right partition 60. However, additional coding parameter candidates may also be removed if they are equal to the coding parameters of left partition 50. However, according to another embodiment of the present invention, encoder 10 can be configured to determine the set of candidate blocks for each second and subsequent partition in coding order by removing from the set of candidate blocks the candidate block or blocks that, when merged with the corresponding partition, would produce one of the supported partition patterns. In some senses, this means the following. The encoder 10 can be configured to determine the merge candidates for a corresponding partition 50 or 60 (i.e., the first and following ones in the coding order) such that each element of the candidate set has exactly one of the previously coded blocks 40 or a partition of the current block 40 associated with the candidate using the corresponding coding parameters of the associated partition. For example, each element of the candidate set can be equal to, i.e., adopt one of these coding parameters from the previously coded partition, or can be derived from at least only the coding parameters of one of the previously coded partitions, e.g., by additional scaling or refinement using additionally transmitted refinement information. However, the encoder 10 can also be configured to accompany the candidate set with further elements or candidates, i.e., coding parameter candidates that have been derived from a combination of coding parameters of more than one previously coded partition, or that have been derived from the coding parameters of a previously coded partition by modification, e.g., by adopting only the coding parameters of a shift parameter list. For "combined" elements, there is no 1:1 correlation between the coding parameters of the corresponding candidate element and those of a corresponding partition. According to a first option illustrated in FIG1 , the encoder 10 can be configured to remove from the overall candidate set all candidates whose coding parameters are equal to the coding parameters of partition 50. According to a later option illustrated in FIG1 , the encoder 10 can be configured to remove only the elements of the candidate set associated with partition 50. Reconciling these two points, the encoder 10 can be configured to remove candidates from the portion of the candidate set that shows a 1:1 correlation to some (e.g., adjacent) previously encoded partition, without extending this removal (and the search for candidates with equal coding parameters) to the rest of the candidate set with the coding parameters obtained by combining. Of course, however, if a combination would also result in redundant representations, this can be resolved by removing the redundant coding parameters from the list or by performing a redundancy check on the combined candidates.

Before describing a decoder embodiment suitable for the embodiment of FIG. 1 just described, an apparatus for encoding, i.e., an encoder, will be described in more detail below with reference to FIG. 2 , based on the embodiment described in more detail with reference to FIG. FIG. 2 shows the encoder as comprising a subdivider 72 configured to subdivide image 20 into blocks 40, a merger 74 configured to merge blocks 40 into one or more sample set groups as described above, an encoder or encoding stage 76 configured to encode image 20 using encoding parameters that vary across image 20 in units of sample set groups, and a bitstream generator 78. Encoder 76 is configured to encode image 20 by predicting image 20 and to encode a prediction residual for a predetermined block. That is, encoder 76 encodes, as described above, not all prediction residuals for blocks 40. Instead, some of these blocks have their skip mode activated. Bitstream generator 78 is configured to embed the prediction residual and coding parameters into bitstream 30, along with one or more syntax elements for each of at least a subset of blocks 40 that signal whether the corresponding block 40 is to be merged with another block into one of the groups and whether the corresponding block uses skip mode. As described above, sub-partitioning information from the sub-partitioning by sub-partitioner 72 may also be used by bitstream generator 78 to encode image 20 into bitstream 30. This is indicated by dashed lines in FIG2 . The merge decision made by merger 74 and the skip mode decision made by encoder 76, as described above, are jointly encoded into bitstream 30 by bitstream generator 78, such that one of the possible states of one or more syntax elements for the current block 40 signals whether the corresponding block is to be merged with another block of image 20 into one of the groups of blocks and has no prediction residual encoded and inserted into bitstream 30. Bitstream generator 78 may, for example, use entropy coding techniques to perform the insertion. Sub-partitioner 72 may be responsible for selecting the sub-partitioning of picture 20 into sub-partitions of block 40, and further sub-partitioning into partitions 50 and 60 accordingly. Merger 74 is responsible for the merge decisions described above, while encoder 76 may, for example, determine the skip mode for block 40. Of course, all of these decisions collectively affect the rate/distortion metric, and thus device 10 may be configured to experiment with a number of decision options to determine which is optimal.

After describing an encoder according to an embodiment of the present invention with reference to Figures 1 and 2, an apparatus for decoding, namely a decoder 80 according to an embodiment, will be described with reference to Figure 3. Decoder 80 of Figure 3 is configured to decode a bitstream 30 in which, as described above, picture 20 is encoded. In particular, decoder 80 is configured to collectively respond to the aforementioned flags within bitstream 30 regarding, for a current set of samples or block 40, a first determination as to whether coding parameters associated with current block 40 are to be set based on a merge candidate or to be obtained from bitstream 30, and a second determination as to whether current block 40 of picture 20 is to be reconstructed solely based on a prediction signal dependent on the coding parameters associated with current block 40 without any residual data, or to be reconstructed using residual data within bitstream 30 by refining a prediction signal dependent on the coding parameters associated with current block 40.

That is, the decoder's functionality largely corresponds to that of the encoder described with respect to Figures 1 and 2. For example, decoder 80 can be configured to perform a sub-division of image 20 into blocks 40. This sub-division can be known to decoder 80, or decoder 80 can be configured to extract the corresponding sub-division information from bitstream 30. Whenever blocks 40 are merged, decoder 80 can be configured to obtain coding parameters associated with block 40 by setting coding parameters based on a merge candidate. To determine the merge candidate, decoder 80 can perform the above-described determination of the merge candidate set or list in exactly the same manner as the encoder. According to some embodiments of the present application, this even includes reducing the initial set/list of merge candidates to avoid redundancy between block partitioning, on the one hand, and block merging, on the other hand, as described above. Whenever merging is activated, selection between the determined merge candidate sets or lists can be performed by decoder 80 by extracting a corresponding merge index from bitstream 30. The merge index indicates the merge candidate to be used from the determined (reduced) merge candidate set or list. Furthermore, as also described above, decoder 80 can also be configured to allow block 40 to be partitioned according to one of the supported partitioning patterns. Of course, one of these partitioning patterns may include a non-partitioning mode, according to which block 40 is not further partitioned. In the case of a flagged scenario where a commonly defined state indicates a detailed description of merge and skip mode activation for a particular block 40, decoder 80 can be configured to reconstruct the current block 40 based solely on the predicted signal, rather than its combination with any residual signal. In other words, in this case, decoder 80 suppresses residual data extraction for the current block 40 and reconstructs the image 20 within the current block 40 solely using a prediction signal derived from the current block's coding parameters. As also described above, decoder 80 can interpret the common state of the flag as a signaling that the current block 40 is an inter-frame predicted block and/or is not to be further partitioned. That is, the decoder 80 may be configured to obtain the coding parameters associated with the current block 40 by setting these coding parameters according to a merge candidate, and to reconstruct the current block 40 of the image 20 based solely on a prediction signal that depends on the coding parameters of the current block 40 without any residual data if a flag in the discussion of the current block 40 within the 30-bitstream signals that the coding parameters associated with the current block 40 are to be set using a merge. However, if the flag in question signals that the current block 40 is not subject to merging or that skip mode is not used, the decoder 80 may be responsive to another flag within the bitstream 30 such that, depending on the other flag, the decoder 80 derives coding parameters associated with the current block by setting the coding parameters according to a corresponding merging candidate, obtains residual data for the current block from the bitstream 30, and reconstructs the current block 40 of the image 20 based on the prediction signal and the residual data, or extracts coding parameters associated with the current block 40 from the bitstream 30, obtains residual data for the current block 40 from the bitstream 30, and reconstructs the current block 40 of the image 20 based on the prediction signal and the residual data. As described above, the decoder 80 may be configured to anticipate the presence of the other flag within the bitstream 30 only if the first flag does not assume a jointly signaling state simultaneously signaling the activation of merge and skip mode. The decoder 80 then extracts the other flag from the bitstream in order to determine whether a merge is to occur without skip mode. Of course, in the case where a second flag signals merge deactivation, while a third flag signals skip mode activation or deactivation, the decoder 80 may alternatively be configured to wait for another third flag within the bitstream 30 for the current block 40 .

2 , FIG4 shows a possible embodiment of an apparatus for decoding of FIG3 . Thus, FIG4 shows an apparatus for decoding, namely, a decoder 80, which includes a subdivider 82 configured to subdivide an image 20 encoded as a bitstream 30 into blocks 40, a merger 84 configured to merge the blocks 40 into groups of one or more blocks, a decoder 86 configured to decode or reconstruct the image 20 using coding parameters that vary across the image 20 in units of sample set groups, and an extractor 88. The decoder 86 is also configured to decode the image 20 for predetermined blocks 40, i.e., those with a skip mode disabled, by predicting the image 20, decoding the prediction residual for the predetermined block 40, and combining the prediction residual with a prediction generated from the predicted image 20. The extractor 88 is configured to extract from the bitstream 30 the prediction residual and the coding parameters, together with one or more syntax elements for each of at least a subset of the blocks 40, which signal whether the corresponding block 40 is to be merged with another block 40 into one of the groups, wherein the merger 84 is configured to perform the merging in response to the one or more syntax elements, wherein one of the possible states of the one or more syntax elements signals that the corresponding block 40 is to be merged with another block 40 into one of the block groups and does not have a prediction residual that is encoded and inserted into the bitstream 30.

Thus, comparing FIG4 with FIG2 , subpartitioner 82 functions similarly to subpartitioner 72, restoring the subpartitions generated by subpartitioner 72. Subpartitioner 82 identifies subpartitions of image 20 by default or by extracting subpartition information from bitstream 30 via extractor 88. Similarly, merger 84 forms a merger of blocks 40 and is activated with respect to blocks 40 and block portions via the aforementioned signaling within bitstream 30. Decoder 86 uses the coding parameters within bitstream 30 to generate a prediction signal for image 20. In the case of a merge, decoder 86 copies the coding parameters of a current block 40 or partition of the current block from a neighboring block/partition or otherwise sets its coding parameters differently depending on the merge candidate.

As described above, extractor 88 is configured to interpret one of the possible states of a flag or syntax element for a current block as simultaneously signaling the activation of merge and skip modes. Simultaneously, extractor 88 may interpret the state as also signaling a predetermined one of the supported partitioning patterns for current block 40. For example, the predetermined partitioning pattern may be a non-partitioning mode, according to which block 40 remains unpartitioned, thus itself forming a partition. Therefore, extractor 88 predicts that bitstream 30 includes partitioning information signaling the partitioning of block 40 only in cases where the corresponding flag or syntax element does not determine the simultaneously signaling state. As will be described in more detail below, the partitioning information may be conveyed within bitstream 30 via a syntax element that simultaneously controls the coding mode of current block 40, i.e., whether partitioning block 40 into frame-coded and intra-frame-coded. In this case, the jointly signaled state of the first flag/syntax element may also be interpreted as signaling an inter-frame prediction coding mode. For each partition generated by the self-signaled partition information, extractor 88 may extract another merge flag from the bitstream in the case where the first flag/syntax element for block 40 does not determine the state of jointly signaling the activation of merging and skipping modes. In this case, skipping mode may necessarily be interpreted by extractor 88 as off, and although merging may be activated accordingly for these partitions using bitstream 30, a residual signal is extracted from bitstream 30 for this current block 40.

3 or 4 is configured to decode the bitstream 30. As described above, the bitstream 30 may signal one of the supported partition patterns for a current block 40 of the image 20. If one of the signaled supported partition patterns specifies a subdivision of the current block 40 into two or more partitions 50 and 60, the decoder 80 may be configured to remove, in coding order 70, from a set of coding parameter candidates for the corresponding partition coding parameter candidates having coding parameters identical or equal to the coding parameters associated with any of these partitions, all partitions except the first partition 50 of these partitions, i.e., partition 60 of the examples shown in FIG1 and FIG3 , wherein these partitions, when combined with the corresponding partition, will form one of the supported partition patterns, i.e., one that is not signaled in the bitstream 30 but is one of the supported partition patterns.

For example, if the number of non-removed coding parameter candidates is non-zero, decoder 80 can be configured to set the coding parameters associated with the corresponding partition 60 depending on one of the non-removed coding parameter candidates. For example, decoder 80 sets the coding parameters of partition 60 to be equal to one of the non-removed coding parameter candidates, with or without further refinement and/or with or without scaling, depending on the temporal distance to which the coding parameters relate. For example, a coding parameter candidate merged with a non-removed candidate may have a reference image index associated with it in addition to the reference image index explicitly signaled within bitstream 30 for partition 60. In this case, the coding parameters of the coding parameter candidates may define motion vectors associated with a corresponding reference image index, and decoder 80 may be configured to scale the motion vector of the last selected non-removed coding parameter candidate according to the ratio between the two reference image indices. Thus, according to the selection just mentioned, the coding parameters that are merged will contain motion parameters, and the reference image indices will be separated therefrom. However, as indicated above, according to further embodiments, these reference picture indices may also be part of the coding parameters subject to merging.

The same applies to the encoders of Figures 1 and 2 and the decoders of Figures 3 and 4 , where the merging behavior can be restricted to inter-prediction blocks 40. Therefore, the decoder 80 and the encoder 10 can be configured to support both intra- and inter-prediction modes for the current block 40, and to perform merging only when the current block 40 is encoded in inter-prediction mode. Consequently, only the previously encoded separate coding/prediction parameters for these inter-prediction blocks can be used to determine/construct the candidate list.

As discussed above, the coding parameters may be prediction parameters, and decoder 80 may be configured to use the prediction parameters for partitions 50 and 60 to derive a prediction signal for the corresponding partitions. Of course, encoder 10 also derives the prediction signal in the same manner. However, encoder 10 additionally sets the prediction parameters and all other syntax elements in bitstream 30 to achieve some optimization in an appropriate optimization sense.

Furthermore, as previously described, the encoder can be configured to insert an index into a (non-removed) coding parameter candidate only if the number of (non-removed) coding parameter candidates for a corresponding partition is greater than 1. Thus, the decoder 80 can be configured to only include, depending on the number of (non-removed) coding parameter candidates (e.g., for partition 60), the syntax element indicating which of the (non-removed) coding parameter candidates was used for the merge if the number of (non-removed) coding parameter candidates is greater than 1. However, in the event that the number of candidate sets becomes smaller than 2, this can generally be eliminated, as described above, by extending the candidate list/set using combined coding parameters, i.e., parameters derived from a combination of coding parameters of more than one—or more than two—previously coded partitions, which limits the reduction of the candidate set to those candidates obtained by taking or deriving from exactly one previously coded partition. The opposite is also possible, i.e., generally removing all coding parameter candidates having the same value for partitions that result in another supported partition pattern.

With regard to the decision, decoder 80 functions similarly to encoder 10. Specifically, decoder 80 can be configured to determine the merging candidate set for a partition of block 40 based on coding parameters associated with previously decoded partitions. In other words, a coding order is defined not only for partitions 50 and 60 of the corresponding block 40, but also for block 40 of image 20 itself. All partitions coded prior to partition 60 can therefore serve as the basis for determining the merging candidate set for any subsequent partition (e.g., partition 60 in the case of FIG. 3 ). As also described above, the encoder and decoder can restrict the determination of the merging candidate set partitions to certain spatial and/or temporal neighbors. For example, decoder 80 can be configured to determine the merging candidate set based on coding parameters associated with previously decoded partitions adjacent to the current partition, where these partitions can be located both outside and inside the current block 40. Of course, the merging candidate determination can also be performed for the first partition in the coding order. Only removal can be performed.

1 , except that one is encoded in an intra-frame prediction mode, the decoder 80 may be configured to determine a set of coding parameter candidates for corresponding non-first partitions 60 from a starting set of previously decoded partitions.

Furthermore, in the case where the encoder introduces sub-partition information into the bitstream to sub-partition the image 20 into blocks 40 , the decoder 80 may be configured to restore the sub-partition of the image 20 into these coded blocks 40 according to the sub-partition information in the bitstream 30 .

With respect to Figures 1-4, it should be noted that the residual signal for current block 40 may be transmitted via bitstream 30 at a granularity that may differ from the granularity defined by the partitioning associated with the coding parameters. For example, for blocks in which skip mode is not enabled, encoder 10 of Figure 1 may be configured to subdivide block 40 into one or more transform blocks, in parallel with, or independently of, the partitioning into partitions 50 and 60. The encoder may signal the corresponding transform block subdivision of block 40 via further subdivision information. Decoder 80 may then be configured to recover the further subdivision of block 40 into one or more transform blocks based on the further subdivision information in the bitstream, and retrieve a residual signal for current block 40 from the bitstream in units of these transform blocks. Transform block partitioning may mean that a transform in the encoder, such as a DCT, and a corresponding inverse transform in the decoder, such as an IDCT, are performed separately within each transform block of block 40. To reconstruct the image 20 within the block 40, the encoder 10 then combines, e.g., adds, the prediction signal derived by applying the coding parameters at the corresponding intervals 50 and 60, and the residual signal, respectively. However, it should be noted that the residual coding may not involve any transformation and inverse transformation, respectively, and that the prediction residual is, for example, encoded in the spatial domain.

Before further describing possible details of further embodiments below, the possible internal structures of the encoder and decoder of Figures 1 through 4 will be described with respect to Figures 5 and 6 , however, the merger and subdivider are not shown in these figures in order to focus on the hybrid coding properties. Figure 5 illustrates an example of how encoder 10 may be internally configured. As shown, encoder 10 may include a subtractor 108, a converter 100, and a bitstream generator 102, which may perform entropy coding, as indicated in Figure 5 . Components 108, 100, and 102 are connected in series between an input 112 for receiving image 20 and an output 114 for outputting the aforementioned bitstream 30. In particular, subtractor 108 has a non-inverting input connected to input 112, and converter 100 is connected between the output of subtractor 108 and a first input of bitstream generator 102, which in turn has an output connected to output 114. 5 further comprises an inverse converter 104 and an adder 110 connected in series to the output of the converter 100 in the stated order. The encoder 10 further comprises a predictor 106 connected between the output of the adder 110 and a further input of the adder 110 and the inverse input of the subtractor 108.

The components of FIG. 5 interact as follows: Predictor 106 predicts a portion of image 20, and the prediction result, i.e., the prediction signal, is applied to the inverting input of subtractor 108. The output of subtractor 108, in turn, represents the difference between the prediction signal and the corresponding portion of image 20, i.e., the residual signal. The residual signal undergoes transform coding in converter 100. That is, converter 100 may perform a transform, such as a DCT or the like, and a subsequent quantization on the transformed residual signal (i.e., transform coefficients) to obtain transform coefficient levels. Inverse converter 104 reconstructs the final residual signal output by converter 100 to obtain a reconstructed residual signal corresponding to the residual signal input to converter 100 (except for information lost due to quantization in converter 100). The reconstructed residual signal is added to the prediction signal output by predictor 106, resulting in a reconstruction of the corresponding portion of image 20, and the output of adder 110 is passed to the input of predictor 106. The predictor 106 operates in various modes as described above, such as an intra-frame prediction mode, an inter-frame prediction mode, and the like. The prediction mode and corresponding coding or prediction parameters applied by the predictor 106 to obtain the prediction signal are transmitted by the predictor 106 to the entropy encoder 102 for insertion into the bitstream.

A possible embodiment of the internal structure of the decoder 80 of Figures 3 and 4, corresponding to the possible embodiment of the encoder shown in Figure 5, is shown in Figure 6. As shown therein, the decoder 80 may include a bitstream extractor 150, which, as shown in Figure 6, is implemented as an entropy decoder, an inverse transformer 152, and an adder 154, which are, in that order, connected between the decoder input 158 and the output 160. Furthermore, the decoder of Figure 6 includes a predictor 156 connected between the output of the adder 154 and its further input. The entropy decoder 150 is connected to the parameter input of the predictor 156.

Briefly describing the decoder functionality of FIG6 , entropy decoder 150 is used to extract all information contained in bitstream 30. The entropy coding mechanism used can be variable length coding or arithmetic coding. Thus, entropy decoder 150 recovers the transform coefficient levels representing the residual signal from the bitstream and transmits them to inverse converter 152. Furthermore, entropy decoder 150 acts like extractor 88 described above and recovers all coding modes and associated coding parameters from the bitstream and transmits them to predictor 156. Separation information and merging information are also extracted from the bitstream by extractor 150. Inverse conversion is performed, i.e., the reconstructed residual signal and the prediction signal derived by predictor 156 are combined, e.g., added, by adder 154. The reconstructed signal is then output at output 160 and transmitted to predictor 156.

As is clear from a comparison of FIG. 5 and FIG. 6 , components 152 , 154 , and 156 correspond functionally to components 104 , 110 , and 106 of FIG. 5 .

In the description of Figures 1 to 6 above, many different possibilities have been presented regarding possible subdivisions of the image 20 and the corresponding granularity of some parameter variations included in the encoded image 20. This possibility is again explained with reference to Figures 7A and 7B. Figure 7A shows a portion of the image 20. According to the embodiment of Figure 7A, the encoder and decoder are configured to first subdivide the image 20 into root blocks 200. This root block is shown in Figure 7A. The subdivision of the image 20 into root blocks is regularly done in columns and rows, as shown by the dotted lines. The size of the root blocks 200 can be selected by the encoder and signaled to the decoder using the bitstream 30. Alternatively, the size of these root blocks 200 can be fixed using a predetermined value. These root blocks 200 are subdivided using a quadtree partitioning to produce blocks 40 consistent with the above, which can be called coding blocks or coding units. These coding blocks or coding units are drawn in Figure 7A using thin solid lines. By doing so, the encoder accompanies each root block 200 with sub-partition information and embeds this sub-partition information into the bitstream. This sub-partition information indicates how the root block 200 is sub-partitioned into blocks 40. The prediction mode varies within the image 20 at the granularity and unit of these blocks 40. As indicated above, each block 40—or each block having a certain prediction mode, such as inter-frame prediction mode—is accompanied by partition information regarding which supported partition pattern is used for the corresponding block 40. However, recalling the aforementioned flags/syntax elements, when signaling a state collectively, one of the supported partition modes for the corresponding block 40 may also be signaled simultaneously, so that explicit transmission of the other partition information for this block 40 can be suppressed and not expected at the encoder end, and thus, also at the decoder end. In the scenario shown in FIG. 7A , for many coding blocks 40, the non-partition mode has been selected, so that the coding blocks 40 spatially coincide with the corresponding partitions. In other words, a coding block 40 is, at the same time, a partition having a corresponding set of prediction parameters associated with it. The classification of the prediction parameters, in turn, depends on the mode associated with the corresponding coding block 40. Other coding blocks, however, are shown by way of example to be further partitioned. The coding block 40 at the top right-hand corner of the root block 200 is, for example, shown to be partitioned into four partitions, whereas the coding block at the bottom right-hand corner of the root block 200 is shown by way of example to be vertically subdivided into two partitions. The subdivisions used to partition into the plurality of partitions are shown with dotted lines. FIG7A also shows the coding order between the plurality of partitions thus defined. As shown, a depth-first pass order is used. Across the root block edges, the coding order can be continued in a scan order according to which the columns of the root block 200 are scanned column-wise from the top to the bottom of the image 20. By this measure, it is possible to have a maximum chance that a partition has a previously coded partition adjacent to its top edge as well as to its left-hand edge. Each block 40—or each block having a certain prediction mode, such as inter-frame prediction mode—can have a merge switch indicator within the bitstream indicating whether merging is enabled for the corresponding partition. It should be noted that the partitioning of a block into partitions/prediction units can be limited to a partition of a maximum of two partitions, with the only exception being the minimum possible block size of block 40. In the case of using quadtree subpartitioning to obtain block 40, this avoids redundancy between subpartition information for subpartitioning image 20 into block 40 and partition information for subpartitioning block 40 into partitions. Alternatively, partitioning into only one or two partitions is permitted, with or without asymmetric partitioning.

7B shows a sub-partition tree. The sub-partition of the root block 200 is shown by solid lines, while the dashed lines mark the partitioning of the leaf blocks of the quadtree sub-partition, which are coding blocks 40. That is, the partitioning of the coding blocks represents an extension of the quadtree sub-partition.

As mentioned above, each coding block 40 may be subdivided into transform blocks in parallel, so that a transform block may represent a different subdivision of the corresponding coding block 40. For each of these transform blocks, which are not shown in FIG. 7 a and FIG. 7 b , a transform of the residual signal of the transform coding block may be performed separately.

In the following, further embodiments of the present invention will be described. While the above embodiments focus on the relationship between block merging on the one hand and block separation on the other hand, the following description also contains the present application's arguments regarding other coding principles known in current codecs, such as skip/direct mode. However, the subsequent description should not be regarded as describing only separate embodiments, that is, the separate embodiments described above. Of course, the following description also discloses possible implementation details of the above-mentioned embodiments. Therefore, the following description uses the reference symbols of the previously described figures, so that a corresponding possible implementation described below will also define possible variations of the above-mentioned embodiments. Most of these variations can be transferred to the above-mentioned embodiments separately.

In other words, embodiments of the present application describe methods for reducing side information rate in image and video coding applications by combining signaling of merging with the absence of residual data for a sample set. In other words, by combining syntax elements indicating the use of a merging mechanism with syntax elements indicating the absence of residual data, side information rate is reduced in image and video coding applications.

Furthermore, before explaining these variations and further details, an overview of image and video codecs is presented.

In image and video coding applications, the sample array associated with a picture is typically partitioned into specific sets of samples (or sample sets), which can represent rectangular or square blocks or any other set of samples encompassing arbitrarily shaped regions, triangles, or any other shape. The subdivision of the sample array can be fixed via syntax or the subdivision can be (at least partially) signaled within the bitstream. To keep the side information rate used to signal the subdivision low, the syntax typically allows only a limited number of choices that result in simple partitioning (e.g., subdividing a block into smaller blocks). A commonly used partitioning mechanism is to partition a square block into four smaller square blocks, or into two rectangular blocks of equal size, or into two rectangular blocks of different sizes, where the actual partitioning employed is signaled within the bitstream. Sample sets are associated with specific coding parameters, which can specify prediction information or residual coding modes, among other things. In video coding applications, partitioning is often performed for motion rendering purposes. All samples of a block (within a partition pattern) are associated with the same set of motion parameters, which may include parameters specifying the prediction type (e.g., list 0, list 1, or bidirectional prediction; and/or transfer or affine prediction, or prediction with a different motion model), parameters specifying the reference images used, parameters specifying the motion relative to these reference images (e.g., displacement vectors, affine motion parameter vectors, or motion parameter vectors for any other motion model), which are typically sent to the predictor as a delta, parameters specifying the accuracy of the motion parameters (e.g., half-sample or quarter-sample accuracy), parameters specifying the weighting of the reference sample signals (e.g., for illumination compensation purposes), or parameters specifying the interpolation filter used to derive the motion-compensated prediction signal for the current block. For each sample set, corresponding coding parameters (e.g., for specifying prediction and/or residual coding) are provided. To achieve improved coding efficiency, the present invention proposes a method and specific embodiments for merging two or more sample sets into so-called sample set groups. All sample sets in this group share the same coding parameters, which can be sent together with one of the sample sets in the group. By doing so, the coding parameters do not need to be sent separately for each sample set in the sample set group, but instead the coding parameters are sent only once for the entire group of sample sets.

Thus, the side information rate for transmitting coding parameters is reduced and the overall coding efficiency is improved. As another approach, an additional refinement of one or more coding parameters can be transmitted for one or more sample sets in a sample set group. This refinement can be applied arbitrarily to all sample sets in a group or only to the sample sets to be transmitted.

Some embodiments of the present invention combine a merging process with a partitioning of a block into sub-blocks 50, 60 (as described above). Typically, image or video coding systems support various partitioning patterns for blocks 40. For example, a square block may not be partitioned, or it may be partitioned into four square blocks of equal size, or into two rectangular blocks of equal size (where the square block can be partitioned vertically or horizontally), or into rectangular blocks of different sizes (horizontally or vertically). These example partitioning patterns are shown in FIG8. In addition to the above, partitioning can even include more than one level of partitioning. For example, square sub-blocks can optionally be further partitioned using the same partitioning pattern. When this partitioning process is combined with a merging process (which allows a (square or rectangular) block to be merged with, for example, one of its neighboring blocks), the issue arises that the same resulting partitioning can be achieved using different combinations of partitioning patterns and merging signals. Thus, the same information can be sent in the bitstream using different codewords, which is clearly near-optimal for coding efficiency. As a simple example, consider a square block that is not further partitioned (as shown in the top left corner of Figure 8). This partitioning can be signaled directly by transmitting a syntax element indicating that the block 40 is not subdivided. However, the same pattern can also be signaled by transmitting a syntax element specifying that the block is subdivided, for example, into two vertically (or horizontally) aligned rectangular blocks 50 and 60. We can then send merge information indicating that the second of these rectangular blocks is merged with the first, resulting in exactly the same partitioning as if we had signaled that the block was not further partitioned. This can also be achieved by first indicating that the block is subdivided into four square subblocks and then sending merge information that effectively merges all four blocks. This concept is clearly near-optimal (because we have different codewords for signaling the same event).

Some embodiments of the present invention reduce side information rate and thus increase coding efficiency for a combination of merging concepts and providing different partitioning patterns for a block. Looking at the partitioning pattern example in FIG8 , when we prohibit (i.e., exclude the bitstream syntax from indicating) the merging of a rectangular block with a first rectangular block, the "simulation" of a block being further partitioned without any partitioning pattern having two rectangular blocks can be avoided. Looking deeper into this topic, it is also possible to "simulate" a pattern that is not sub-partitioned by merging the second rectangular block with any other neighbor (i.e., not the first rectangular block) that is associated with the same parameters (e.g., information for specifying prediction) as the first rectangular block. Redundancy can be avoided by regulating the way merging information is transmitted so that when these merging parameters result in a pattern that can also be achieved by signaling support for one of the partitioning patterns, the transmission of specific merging parameters does not include bitstream syntax. As an example, if the current partition pattern indicates a sub-partition into two rectangular blocks, as shown in Figures 1 and 3 , before sending merge information for the second block, i.e., block 60 in the case of Figures 1 and 3 , it can be checked which possible merge candidates have the same parameters (e.g., parameters indicating the prediction signal) as the first rectangular block, i.e., block 50 in the case of Figures 1 and 3 . All candidates with the same motion parameters (including the first rectangular block itself) are removed from the merge candidate set. The codeword or flag sent to signal the merge information is applicable to the generated candidate set. If the candidate set becomes empty due to the parameter check, no merge information can be sent. If the candidate set contains exactly one entry, it only signals whether the block is merged, but the candidate does not need to be signaled because it can be derived at the decoder. The same concept as in the above example can also be applied to a partition pattern that partitions a square block into four smaller square blocks. Here, the transfer of merge flags is applicable in that neither a partition pattern indicating no subdivision nor two partition patterns indicating subdivision into two rectangular blocks of equal size can be implemented using a combination of merge flags. Although the above examples illustrate most concepts with specific partition patterns, it should be understood that the same concepts (avoiding the description of a specific partition pattern through a combination of another partition pattern and corresponding merge information) can be applied to any other set of partition patterns.

Another point to consider is that the merging concept is similar in some ways to the skip or direct modes found in video coding designs. In skip/direct mode, no motion parameters are sent for a current block at all, but rather are inferred from spatial and/or temporal neighbors. In a particularly efficient concept of skip/direct mode, a list of motion parameter candidates (reference frame indices, displacement vectors, etc.) is generated from spatial and/or temporal neighbors, and an index into this list indicating which candidate parameter is selected is sent. For bidirectionally predicted blocks (or multi-hypothesis frames), a separate candidate can be signaled for each reference list. Possible candidates may include the block to the top of the current block, the block to the left of the current block, the block to the left of the top of the current block, the block to the right of the top of the current block, intermediate predictors of various candidates, and co-located blocks in one or more previous reference frames (or any other previously coded blocks, or combinations thereof).

Combining skip/direct with the merge concept means that a block can be coded using either skip/direct or merge mode. While the skip/direct and merge concepts are similar, there are differences between the two concepts, which will be explained in more detail in Section 1. The main difference between skip and direct is that skip mode further signals that no residual signal is sent. When the merge concept is used, a flag is typically sent that signals whether a block contains non-zero transform coefficient levels.

To achieve improved coding efficiency, the embodiments described above and below combine signaling of whether a sample set uses the coding parameters of another sample set with signaling of whether no residual signal is sent for the block. The combined flag indicates that a sample set uses the coding parameters of another sample set and no residual data is sent. For this case, only one flag, not two, needs to be sent.

As described above, some embodiments of the present invention also provide an encoder with greater freedom in generating a bitstream, as the merging method significantly increases the number of possibilities for selecting a partitioning of the image sample array without introducing redundancy into the bitstream. Because the encoder can choose between more options, for example, to minimize a specific rate/distortion metric, coding efficiency can be improved. As an example, several additional patterns (e.g., the pattern in FIG9 ) that can be represented using a combination of sub-partitioning and merging can be tested (using corresponding block sizes for motion estimation and mode decision), and the best pattern provided using pure partitioning ( FIG8 ) and partitioning and merging ( FIG9 ) can be selected based on a specific rate/distortion metric. Furthermore, each block can be tested to see whether merging with any set of previously coded candidates yields a specific rate/distortion metric reduction, and then the corresponding merge flag can be set during the encoding process. In summary, there are many possibilities for operating an encoder. In a simple approach, the encoder can first determine the optimal sub-partitioning of the sample array (as in current encoding schemes). Next, it is checked for each sample set whether merging with another sample set or another group of sample sets reduces a specific rate/distortion cost metric. At this point, the prediction parameters associated with the merged sample set group can be re-estimated (e.g., by performing a new motion search), or the prediction parameters for the previously determined merge of the current sample set and the candidate sample set (or sample set group) can be estimated for the considered sample set group. In a more general approach, a specific rate/distortion cost metric can be estimated for additional candidate groups of sample sets. As a specific example, when testing various possible partitioning patterns (e.g., see FIG8 ), some or all patterns represented using combinations of partitioning and merging (e.g., see FIG9 ) can also be tested. That is, a specific motion estimation and mode decision process is performed for all patterns, and the pattern that produces the lowest rate/distortion metric is selected. This process can also be combined with the low-complexity process as described above, so that for the generated block it is additionally tested whether merging with previously coded blocks (eg outside the diagrams of Figures 8 and 9) results in a reduction in the rate/distortion metric.

Below, some possible detailed implementations of the above-described embodiments are described, for example, for the encoders shown in Figures 1, 2, and 5 and the decoders shown in Figures 3, 4, and 6. As noted above, the same can be applied to image and video coding. As described above, a picture, or a specific set of sample arrays for a picture, can be decomposed into blocks, which are associated with specific coding parameters. These pictures typically contain multiple sample arrays. In addition, a picture may also be associated with additional auxiliary sample arrays, for example, to specify transparency information or a depth map. The picture sample arrays (including auxiliary sample arrays) can be grouped into one or more so-called plane groups, where each plane group contains one or more sample arrays. The plane groups of a picture can be coded independently or, if a picture is associated with more than one plane group, predicted from other plane groups of the same picture. Each plane group is typically decomposed into blocks. These blocks (or corresponding blocks of sample arrays) are predicted using either inter-picture prediction or intra-picture prediction. These blocks can have different sizes and can be square or rectangular. The division of a picture into blocks can be fixed using syntax or can be (at least partially) signaled within the bitstream. Typically, syntax elements are sent that signal the subdivision of blocks of a predetermined size. These syntax elements can specify whether and how a block is to be subdivided into smaller blocks and are associated with coding parameters, such as those used for prediction. An example of a possible partitioning pattern is shown in FIG8 . The decoding of the associated coding parameters for all samples of a block (or corresponding blocks of an array of samples) is specified in some manner. In this example, all samples of a block are predicted using the same set of prediction parameters, such as a reference index (which identifies a reference picture from a set of previously coded pictures), motion parameters (which specify the measure of motion of the block between the reference picture and the current picture), parameters specifying interpolation filters, intra-frame prediction modes, etc. Motion parameters can be represented using displacement vectors with horizontal and vertical components or using higher-order motion parameters, such as affine motion parameters comprising six components. It is also possible for more than one set of specific prediction parameters (e.g., reference indices and motion parameters) to be associated with a single block. Therefore, for each set of these specific prediction parameters, a single intermediate prediction signal is generated for the block (or the corresponding block of sample arrays), and the final prediction signal is constructed using a combination of these intermediate prediction signals. The corresponding weighting parameters and possibly a fixed offset (which is added to the weighted sum) can be fixed for a picture, a reference picture, or a set of reference pictures, or they can be included in the prediction parameter set for the corresponding block. The difference between the original block (or the corresponding block of sample arrays) and its prediction signal, also known as the residual signal, is typically transformed and quantized. Typically, a two-dimensional transform is applied to the residual signal (or the corresponding sample array for the residual block). For transform coding, the blocks (or corresponding blocks of sample arrays) for which a specific set of prediction parameters has been used can be further separated before the transform is applied. These transform blocks can be equal to or smaller than the block used for prediction. It is also possible for a transform block to contain more than one block used for prediction. Different transform blocks can have different sizes and can represent square or rectangular blocks. In the examples above for Figures 1-7, it should be noted that the leaf node, which may be initially subdivided, i.e., coding block 40, can, on the one hand, be further subdivided in parallel into partitions of a defined coding parameter granularity, and, on the other hand, a two-dimensional transform is applied to each transform block. After the transformation, the resulting transform coefficients are quantized and so-called transform coefficient levels are obtained. These transform coefficient levels, along with the prediction parameters and, if present, the subdivision information, are entropy coded. In particular, the coding parameters for these transform blocks are referred to as residual parameters. These residual parameters, along with the prediction parameters and, if present, the subdivision information, can be entropy coded. In the current H.264 video coding standard, a flag called the Coding Block Flag (CBF) can signal that all transform coefficient levels are zero and, therefore, no residual parameters are coded. According to the present invention, this signaling is combined into merged activation signaling.

In current image and video coding standards, the possibilities for subdividing a picture (or plane group) into blocks are very limited using syntax. Typically, it only specifies whether (and how) a block of a predetermined size can be subdivided into smaller blocks. For example, the maximum block size in H.264 is 16×16. 16×16 blocks are also called macroblocks, and each picture is divided into macroblocks in the first stage. For each 16×16 macroblock, it can be signaled whether it is encoded as one 16×16 block, two 16×8 blocks, two 8×16 blocks, or four 8×8 blocks. If a 16×16 block is subdivided into four 8×8 blocks, each of these 8×8 blocks can be encoded as one 8×8 block, two 8×4 blocks, two 4×8 blocks, or four 4×4 blocks. The advantage of specifying a small set of possible partitions into blocks in current image and video coding standards is that the side information rate used to signal the sub-partitioning can be kept low. However, a disadvantage is that the bit rate required to transmit prediction parameters for the block becomes significant, as explained below. The side information rate used to signal the prediction information often represents a significant amount of the overall bit rate for a block. When this side information is reduced, for example, by using larger block sizes, coding efficiency can be increased. Compared to H.264, it is also possible to increase the set of supported partition patterns. For example, the partition pattern shown in FIG8 can be provided for square blocks of all (or selected) sizes. Actual images or pictures of a video sequence include arbitrarily shaped objects with specific properties. For example, these objects or object parts are characterized by unique texture structures or unique motion. Typically, the same set of prediction parameters can be applied to these objects or object parts. However, object boundaries typically do not coincide with possible block boundaries of large prediction blocks (e.g., 16×16 macroblocks in H.264). An encoder typically determines the subdivision (from a finite set of possible subdivisions) that minimizes a particular rate/distortion cost metric. For arbitrarily shaped objects, this can result in a large number of small blocks. This can also maintain fidelity when more partitioning patterns (as described above) are provided. It should be noted that the number of partitioning patterns should not be too large, as this would require a lot of side information and/or encoder/decoder complexity to signal and process these patterns. Therefore, arbitrarily shaped objects typically result in a large number of small blocks due to their partitioning. And because each of these small blocks is associated with a set of prediction parameters that must be transmitted, the side information rate can become a major component of the overall bit rate. However, because many small blocks still represent regions of the same object or object part, the prediction parameters for some of the resulting blocks are identical or very similar. Intuitively, coding efficiency can be increased if the syntax is extended in a way that not only allows subdivision of a block, but also allows merging of two or more resulting blocks. Thus, we obtain groups of blocks encoded with the same prediction parameters. The prediction parameters for this group of blocks only need to be encoded once. In the example of Figures 1-7 above, for example, if a merge occurs, the coding parameters for the current block 40 are not transmitted. That is, the encoder does not transmit the coding parameters associated with the current block, and the decoder does not expect the bitstream 30 to contain the coding parameters for the current block 40. Instead, according to certain embodiments thereof, only the refined information may be transmitted for the merged current block 40. The candidate set and its reduction, as well as the decision on merges and others, are also performed simultaneously for the other coding blocks 40 of the image 20. These coding blocks typically form coding block groups and coding chains, where the coding parameters for these groups are only fully transmitted once in the bitstream.

If the bitrate saved by reducing the number of coded prediction parameters is greater than the bitrate otherwise spent encoding the merge information, then the merge described above results in increased coding efficiency. It should be further noted that the syntax extension described above (for merge) provides the encoder with additional freedom to choose how to partition a picture or plane group into blocks without introducing redundancy. The encoder is not restricted to first subdividing and then checking whether some of the resulting blocks have the same set of prediction parameters. As a simple alternative, the encoder can first determine the subdivision, as in current coding techniques. And then, for each block, check whether merging with one of its neighboring blocks (or a previously determined group of blocks) reduces the bitrate/distortion cost metric. In this case, the prediction parameters associated with the new group of blocks can be re-estimated (e.g., by performing a new motion search) or the previously determined prediction parameters for the current block and neighboring blocks or groups of blocks can be estimated for the group of blocks. An encoder can also directly examine the pattern (or a subset thereof) provided by a combination of slicing and merging; that is, motion estimation and mode decision can be performed based on the generated shape. The merging information can be signaled on a block basis. Effectively, the merging can also be interpreted as an inference of the prediction parameters for the current block, where these inferred prediction parameters are set equal to the prediction parameters of one of the neighboring blocks.

For modes other than skip, additional flags, similar to CBF, are required to signal that no residual signal is being sent. The current video coding standard H.264 has two different skip/direct modes, selected at the picture level: temporal direct mode and spatial direct mode. Both direct modes are applicable only to B-pictures. In temporal direct mode, the reference index for reference picture list 0 is set to 0, and the reference index for reference picture list 1 and the motion vectors for both reference lists are derived based on the motion data of the co-located macroblock of the first reference picture in reference picture list 1. Temporal direct mode uses the motion vector from the temporally co-located macroblock and adjusts the motion vector size based on the temporal distance between the current and co-located macroblocks. In spatial direct mode, the reference index and motion vectors for both reference picture lists are derived primarily based on spatially neighboring motion data. The reference index is selected as the minimum of the corresponding reference indices among the spatial neighbors, and each motion vector component is set equal to the median of the corresponding motion vector components among the spatial neighbors. Skip mode can only be used to encode 16×16 macroblocks (in P and B pictures) in H.264, and direct mode can be used to encode 16×16 macroblocks or 8×8 sub-macroblocks. In contrast to direct mode, if merging is applied to the current block, all prediction parameters are copied from the current block and the merged block. Merging can also be applied to arbitrary block sizes, resulting in the more flexible partitioning of patterns described above, where all samples in a pattern are predicted using the same prediction parameters.

The basic idea described in the embodiments of the present invention is to reduce the bit rate required to send the CBF flag by combining combining and CBF flag.If a sample set uses combining and no residual data is sent, then one flag is sent to signal both.

To reduce the side information rate in image and video coding applications, specific sample sets (which can represent rectangular or square blocks, arbitrarily shaped regions, or any other sample sets) are typically associated with a specific set of coding parameters. Coding parameters are included in the bitstream for each of these sample sets. These coding parameters can represent prediction parameters that specify how the corresponding sample is predicted using previously coded samples. The partitioning of an array of image samples into sample sets can be fixed using syntax or signaled using corresponding sub-partitioning information within the bitstream. Multiple partitioning patterns are allowed for a block. Coding parameters for a sample set are sent in a predetermined order, given by syntax. They can be signaled for a current sample set that is merged with one or more other sample sets (e.g., for prediction purposes) to form a sample set group. The set of possible values for the corresponding merging information can be adapted to the partitioning pattern used, such that a particular partitioning pattern cannot be represented using a combination of other partitioning patterns and corresponding merging data. Coding parameters for a sample set group need only be sent once. In addition to the prediction parameters, residual parameters (e.g., transform and quantization side information and transform coefficient levels) can be sent. If the current sample set is merged, side information describing the merging process is sent. This side information will further be referred to as merge information. Embodiments of the present invention describe a concept that combines the signaling of merge information with the signaling of a coded block flag (indicating whether residual data is present for a block).

In a specific embodiment, the merge information includes a combined flag, called mrg_cbf, which is equal to 1 if the current sample set is merged and no residual data is sent. In this case, no further coding parameters and residual parameters are sent. If the combined mrg_cbf flag is equal to 0, another flag is encoded indicating whether merging is applied. A further flag is encoded indicating that no residual parameters are sent. In CABAC and context-adaptive VLC, the context for the derivation of the probability of syntax elements related to the merge information (and VLC list switching) can be selected as a function of previously sent syntax elements and/or decoded parameters (e.g., the combined mrg_cbf flag).

In a preferred embodiment, the merge information including the combined mrg_cbf flag is encoded before encoding parameters (eg, prediction information and sub-partition information).

In a preferred embodiment, the merge information including the combined mrg_cbf flag is encoded after encoding the parameter subsets (eg prediction information and sub-partition information). For each sample set, generated from the sub-partition information, the merge information can be encoded.

In the embodiments described below with reference to Figures 11 to 13, mrg_cbf is referred to as skip_flag. In general, mrg_cbf may be referred to as merge_skip to indicate that it is another skip version of block merging.

The following preferred embodiments are described for sample sets representing rectangular and square blocks, but are straightforward to extend to arbitrarily shaped regions or other sample sets. The preferred embodiments describe a combination of syntax elements for a merge mechanism and a syntax element indicating no residual data. Residual data includes residual side information and transform coefficient levels. For all preferred embodiments, the absence of residual data is indicated using a coded block flag (CBF), but other representations or flags are also possible. A CBF equal to 0 refers to a situation in which no residual data is sent.

1. Combination of Merge Markers and CBF Markers

Hereinafter, the auxiliary merge activation flag is referred to as mrg, and later, with respect to Figures 11 to 13, the same is referred to as merge_flag. Similarly, the merge index is currently referred to as mrg_idx, and later merge_idx is used.

The possible combinations of the merge flag and the CBF flag using a syntax element are described in this section. The description of these possible combinations described below can be transferred to any of the above descriptions shown in Figures 1 to 6.

In a preferred embodiment, up to three syntax elements are sent to indicate the merge information and the CBF.

The first syntax element, referred to below as mrg_cbf, indicates whether the current set of samples is merged with another set of samples and whether all corresponding CBFs are equal to 0. The mrg_cbf syntax element can only be encoded if the derived set of a candidate sample set is not the empty set (after removing candidates that would produce a partition that can be signaled using a different partition pattern without merging). However, it can be guaranteed by the original value that the merge candidate list does not disappear, that at least one or even at least two merge candidates are available. In a preferred embodiment of the present invention, the mrg_cbf syntax element is encoded if the derived set of a candidate sample set is not the empty set, as described below.

If the current block is merged and the CBF is equal to 0 for all components (eg, one luma and two chroma components), the mrg_cbf syntax element is set to 1 and encoded.

Otherwise, the mrg_cbf syntax element is set equal to 0 and is coded.

The values 0 and 1 for the mrg_cbf syntax element may also be toggled.

The second syntax element, further referred to as mrg, indicates whether the current sample set is to be merged with another sample set. If the mrg_cbf syntax element is equal to 1, the mrg syntax element is not coded and is inferred to be equal to 1. If the mrg_cbf syntax element is not present (because the derived candidate sample set is an empty set), the mrg syntax element is also not present but is inferred to be equal to 0. However, it can be guaranteed by the original value that the merge candidate list does not disappear, that at least one or even at least two merge candidates are available.

A third syntax element, further referred to as mrg_idx, is encoded only if the mrg syntax element is equal to 1 (or is inferred to be equal to 1) and indicates which set of candidate sample sets is used for merging. In a preferred embodiment, the mrg_idx syntax element is encoded only if a derived set of a candidate sample set contains more than one candidate sample set. In a further preferred embodiment, the mrg_idx syntax element is encoded only if at least two sample sets of a derived set of a candidate sample set are associated with different coding parameters.

It should be mentioned that the merge candidate list can be fixed to decouple analysis and reconstruction, thereby improving analysis throughput and making it more robust to information loss. For greater precision, decoupling can be determined using a fixed configuration of list items and codewords. This eliminates the need for a fixed list length. However, adding additional candidates while fixing the list length allows compensating for the coding efficiency loss of a fixed (longer) codeword. Therefore, as described above, the merge index syntax element can only be sent if the candidate list contains more than one candidate. However, this would require deriving the list before analyzing the merge index, preventing the two procedures from being performed in parallel. To increase analysis throughput and make the analysis procedure more robust to transmission errors, this dependency can be removed by using a fixed codeword and a fixed number of candidates for each index value. If this number cannot be achieved using candidate selection, auxiliary candidates may be derived to complete the list. These additional candidates may include so-called combined candidates, which are created from the motion parameters of potentially different candidates already in the list, as well as a zero motion vector.

In a preferred embodiment, the merging information for a sample set is encoded after a subset of prediction parameters (or, more generally, specific coding parameters associated with the sample set) has been transmitted. The prediction parameter subset may include one or more reference picture indices or one or more motion parameter vector components or a reference picture index and one or more motion parameter vector components, etc.

In a preferred embodiment, the mrg_cbf syntax element for merge information is encoded only for a reduced set of partitioning modes. A possible set of partitioning modes is shown in FIG8 . In a preferred embodiment, this reduced set of partitioning modes is limited to one and corresponds to the first partitioning mode (top left of the table in FIG8 ). As an example, mrg_cbf is encoded only if a block is not further partitioned. As a further example, mrg_cfb can be encoded only for square blocks.

In another preferred embodiment, the mrg_cbf syntax element of the merge information is encoded only for one block of a partition, where this partition is shown in FIG8 for one possible partition pattern, e.g., a partition pattern with four left bottom blocks. In a preferred embodiment, if more than one block is merged using one of these partition patterns, the merge information for the first merged block (in decoding order) contains the mrg_cbf syntax element for a complete partition. For all other blocks of the same partition pattern that are subsequently decoded, the merge information only contains the mrg syntax element indicating whether the current sample set was merged with another sample set. Whether residual data is present is inferred from the encoded mrg_cbf syntax element in the first block.

In a further preferred embodiment of the present invention, merge information for a sample set is encoded before the prediction parameters (or, more generally, the specific coding parameters associated with the sample set). The merge information, including the mrg_cbf, mrg, and mrg_idx syntax elements, is encoded in the manner described above for the first preferred embodiment. These prediction or coding parameters and the residual parameters are only sent if the merge information signals that the current sample set is not merged with another sample set and the CBF is equal to 1 for at least one component. In a preferred embodiment, if the mrg_cbf syntax element indicates that the current block is merged and the CBF is equal to 0 for all components, no further signaling is required for the current block after the merge information.

In another preferred embodiment of the present invention, the syntax elements mrg_cbf, mrg, and mrg_idx are combined and encoded as one or two syntax elements. In one preferred embodiment, mrg_cbf and mrg are combined into a syntax element that indicates any of the following: (a) the block is merged and does not contain residual data, (b) the block is merged and contains residual data (or may contain residual data), or (c) the block is not merged. In another preferred embodiment, the syntax elements mrg and mrg_idx are combined into a syntax element. If N is the number of merge candidates, the combined syntax element indicates one of the following: the block is not merged, the block is merged with candidate 1, the block is merged with candidate 2, ..., the block is merged with candidate N. In a further preferred embodiment of the present invention, the syntax elements mrg_cfb, mrg, and mrg_idx are combined into a syntax element that indicates one of the following (N is the number of candidates): the block is not merged, the block is merged with candidate 1 and does not contain residual data, the block is merged with candidate 2 and does not contain residual data, ..., the block is merged with candidate N and does not contain residual data, the block is merged with candidate 1 and contains residual data, the block is merged with candidate 2 and contains residual data, ..., the block is merged with candidate N and contains residual data. The combined syntax element can be sent using variable length coding, arithmetic coding, or binary arithmetic coding using any specific binary mechanism.

2. Combination of Merge Marker and CBF Marker and Skip/Direct Mode

Skip/direct mode can be supported for all or only certain block sizes and/or block shapes. In the skip/direct mode extension described in the current video coding standard H.264, a set of candidate blocks is used for skip/direct mode. The difference between skip and direct modes is whether residual parameters are transmitted. These skip and direct parameters (e.g., for prediction) can be inferred from either of the corresponding candidates. A candidate index is encoded that signals which candidate is used to infer these coding parameters. If multiple predictions are combined to form the final prediction signal for the current block (such as for bidirectionally predicted blocks used in H.264 B-frames), each prediction can involve a different candidate. Therefore, a candidate index can be encoded for each prediction.

In a preferred embodiment of the present invention, the skip/direct candidate list may include different candidate blocks from the merge mode candidate list. An example is shown in FIG10 . The candidate list includes the following blocks (the current block is denoted by Xi):

Movement vector (0, 0)

Middle (left, top, between corners)

Left block (Li)

Upper block (Ai)

Corner blocks (in order: upper right (Ci1), lower left (Ci2), upper left (Ci3))

The same block in a different, but previously coded picture

The following symbols are used to describe the following embodiments:

set_mvp_ori is a candidate set used in skip/direct mode. This set consists of {center, left, top, corner, same position}, where center is the middle value (the middle value of the order of left, top, and corner), and same position is given by the closest reference frame (or the first reference picture in one of the reference picture lists) and the corresponding motion vectors are measured based on temporal distance. Motion vectors with both components equal to zero may be inserted into the candidate list, for example if there are no left, no top, and no corner blocks.

set_mvp_comb is a subset of set_mvp_ori.

In a preferred embodiment, both skip/direct mode and block merge mode are supported. Skip/direct mode uses the original candidate set, set_mvp_ori. Merge information about block merge mode may include the combined mrg_cbf syntax element.

In another embodiment, both skip/direct mode and block merge mode are supported, but skip/direct mode uses a modified candidate set, set_mvp_comb. This modified candidate set can be a specific subset of the original set, set_mvp_ori. In a preferred embodiment, the modified candidate set consists of corner blocks and co-located blocks. In a preferred embodiment, the modified candidate set consists only of co-located blocks. Further subsets are also possible.

In a preferred embodiment, the merge information including the mrg_cbf syntax element is encoded before the skip mode related parameters.

In a preferred embodiment, the skip mode related parameters are encoded before the merge information including the mrg_cbf syntax element.

According to another embodiment, direct mode may not be activated (or even presented) and the block merge has a candidate extended set with skip mode replaced by mrg_cbf.

In a preferred embodiment, the candidate list for block merging may include different candidate blocks. An example is shown in FIG10 . The candidate list includes the following blocks (the current block is represented by Xi):

Movement vector (0, 0)

Left block (Li)

Upper block (Ai)

The same block in a different, but previously coded picture

Corner blocks (in order: upper right (Ci1), lower left (Ci2), upper left (Ci3))

Combined bi-prediction candidates

Unscaled bidirectional prediction candidates

It should be noted that the block merging candidate position can be the same as the MVP list in inter-frame prediction to save memory access.

Further, the list may be "fixed" as described above in order to decouple parsing and reconstruction to improve parsing and be more robust to information loss.

3. CBF encoding

In a preferred embodiment, if the mrg_cfb syntax element is equal to 0 (which signals that the block is not merged or contains non-zero residual data), a flag is sent to signal whether all components of the residual data (e.g., one luma and two chroma components) are zero. If mrg_cfb is equal to 1, this flag is not sent. In a specific configuration, if mrg_cfb is equal to 0, this flag is not sent and the mrg syntax element indicates that the block is merged.

In another preferred embodiment, if the mrg_cfb syntax element is equal to 0 (which signals whether the block is not merged or whether it contains non-zero residual data), a separate syntax element for each component is sent signaling whether the residual data for that component is zero.

Different context models can be used with mrg_cbf.

Therefore, the above embodiment describes an apparatus for encoding an image, which comprises

a sub-segmenter configured to sub-segment the image into a sample set of samples;

a merger configured to merge the sample sets into one or more mutually exclusive sets of sample sets;

an encoder configured to encode the picture in units of mutually exclusive sets of the sample sets using coding parameters that vary across the picture, wherein the encoder is configured to encode the picture by predicting the picture and encoding a prediction residual for a predetermined set of samples; and

A bitstream generator is configured to insert the prediction residual and the coding parameters into a bitstream together with one or more syntax elements for each of at least a subset of the sample sets, the syntax elements signaling whether the corresponding sample set is merged with another sample set into one of the mutually exclusive sets.

Furthermore, a device for decoding a bitstream having an image encoded therein has been described, comprising

a sub-segmenter configured to sub-segment the image into a sample set of a plurality of samples;

a merger configured to merge the sample sets into mutually exclusive sets each of which is one or more sample sets;

a decoder configured to decode the picture using coding parameters that vary across the picture in units of mutually exclusive sets of the sample sets, wherein the decoder is configured to decode the picture by predicting the picture for predetermined sets of samples, decoding a prediction residual for the predetermined sets of samples, and combining the prediction residual with a prediction resulting from predicting the picture;

an extractor configured to extract the prediction residual and the coding parameters from the bitstream together with one or more syntax elements for each of at least a subset of the sample sets, the syntax elements signaling whether the respective sample set is to be merged with another sample set into one of the mutually exclusive sets, wherein the merger is configured to perform the merging in response to the syntax elements.

Therein, one of the possible states of one or more syntax elements will signal that the corresponding sample set will be merged with another sample set into one of the mutually exclusive sets and that the corresponding sample set is not coded and the prediction residual is inserted into the bitstream.

The extractor is further configured to extract sub-partitioning information from the bitstream, and the sub-partitioner is configured to sub-partition the image into sets of samples in response to the sub-partitioning information.

The extractor and the merger are configured to sequentially step through the sample sets according to a sample set scanning order, and, for a current sample set,

Extracting a first binary syntax element (mrg_cbf) from the bitstream;

If the first binary syntax element is determined to be one, merging the current sample set into one of the mutually exclusive sets by inferring that coding parameters for the current sample set are equal to coding parameters associated with the mutually exclusive sets, skipping extraction of prediction residuals for the current sample set, and stepping to the next sample set in sample set scanning order;

If the first binary syntax element is determined to be in a second binary state, extracting a second syntax element (mrg, mrg_idx) from the bitstream; and

Depending on the second syntax element, the current sample set is merged into one of the mutually exclusive sets by inferring that the coding parameters for the current sample set are equal to the coding parameters associated with the mutually exclusive sets, or extraction of the coding parameters for the current sample set is performed while extracting at least one further syntax element related to the prediction residual for the current sample set.

The one or more syntax elements for each of at least a subset of the sample sets also signal which set of predetermined candidate sample sets adjacent to the corresponding sample set is to be merged with if the corresponding sample set is to be merged with another sample set into any one of the mutually exclusive sets.

The extractor is also configured to, if one or more syntax elements do not signal that the corresponding sample set is to be merged with another sample set into any mutually exclusive set,

then extracting one or more further syntax elements (skip/direct mode) from the bitstream, which signal which of predetermined candidate sample sets adjacent to the corresponding sample set the corresponding sample set is to be merged with if the corresponding sample set is to be merged with another sample set into any of the mutually exclusive sets.

In this case, the set of predetermined candidate sample sets and the further set of predetermined candidate sample sets may be mutually exclusive or intersecting with respect to a minority of the set of predetermined candidate sample sets and the further set of predetermined candidate sample sets, respectively.

The extractor is also configured to extract sub-partition information from the bitstream, and the sub-partitioner is configured to hierarchically sub-partition the image into sample sets in response to the sub-partition information, and the extractor is configured to sequentially step through a child sample set of a parent sample set, wherein the parent sample set consists of sample sets into which the image is sub-partitioned, and for a current child sample set,

Extracting a first binary syntax element (mrg_cbf) from the bitstream;

If the first binary syntax element is determined to be a first binary state, merging the current subset sample set into one of the mutually exclusive sets by inferring that coding parameters for the current subset sample set are equal to coding parameters associated with the mutually exclusive sets, skipping extraction of prediction residuals for the current subset sample set, and stepping to a next subset sample set;

If the first binary syntax element is determined to be in a second binary state, extracting a second syntax element (mrg, mrg_idx) from the bitstream; and

depending on the second syntax element, merging the current subset sample set into one of the groups by inferring that the coding parameters for the current subset sample set are equal to the coding parameters associated with the group, or performing extraction of the coding parameters for the current subset sample set while extracting at least one further syntax element related to the prediction residual for the current subset sample set, and then stepping to the next subset sample set,

For the next subset sample set, if the first binary syntax element of the current subset sample set is determined to be in the first binary state, skipping the extraction of the first binary syntax element and starting to extract the second syntax element instead; and if the first binary syntax element of the current subset sample set is determined to be in the second binary state, extracting the first binary syntax element.

Assume, for example, that a parent sample set (CU) is partitioned into two child sample sets (PUs). Then, if for the first PU, the first binary syntax element (merge_cbf) has a first binary state, 1) the first PU uses merging and the first and second PUs (the entire CU) have no residual data in the bitstream, and 2) for the second PU, the second binary syntax element (merge_flag, merge_idx) is signaled. However, if the first binary syntax element has a second binary state for the first PU, then 1) for the first PU, the second binary syntax element (merge_flag, merge_idx) is signaled and residual data is in the bitstream, while 2) for the second PU, the first binary syntax element (merge_cbf) is signaled. Therefore, it is possible that merge_cbf is also signaled at the PU level, i.e., for a successive child sample set, if merge_cbf is in the second binary state for all previous child sample sets. If merge_cbf is in the first binary state for a sequential subset of sample sets, then all subset sample sets below this subset do not have residual data in the bitstream. For example, for a CU partitioned into, for example, 4 PUs, merge_cbf may be in the first binary state for the second PU, indicating that the third and fourth PUs in the coding order do not have residual data in the bitstream, but the first PU does or may have it.

The first binary syntax element and the second binary syntax element may be encoded using context-adaptive variable length coding or context-adaptive (binary) arithmetic coding and the context for encoding these syntax elements is derived from values for these syntax elements in previously coded blocks.

As described in another preferred embodiment, the syntax element merge_idx can only be sent if the candidate list contains more than one candidate. This needs to be derived from the merge index before analyzing the list to prevent the two procedures from being performed in parallel. To increase the analysis throughput and make the analysis procedure more robust to transmission errors, it is possible to remove this dependency by using a fixed codeword for each index value and a fixed number of candidates. If this number cannot be reached by a candidate selection, it is possible to derive additional candidates to complete the list. These additional candidates can include so-called combined candidates, which can be created from the motion parameters of possibly different candidates already in the list, as well as the zero motion vector.

In another preferred embodiment, the syntax for signaling which blocks from these candidate sets are used simultaneously at the encoder and decoder. For example, if three options are provided for merging blocks, only those three options appear in the syntax and are considered for entropy coding. The probability of all other options is assumed to be zero, and the entropy codec is adjusted simultaneously at the encoder and decoder.

The prediction parameters inferred as a result of the merging procedure may represent the complete set of prediction parameters associated with a block or they may represent a subset of these prediction parameters (eg, one hypothetical prediction parameter for a block where multiple hypothetical predictions are used).

In a preferred embodiment, syntax elements related to merge information are entropy coded using a context model.

The method for transferring the above-described embodiment to a specific syntax is described below with reference to the following figures. In particular, Figures 11 through 13A and 13B illustrate various portions of a syntax that utilizes the advantages of the above-described embodiment. Specifically, according to the embodiment described below, image 20 is first partitioned upward into coding tree blocks, whose image content is encoded using the syntax coding_tree shown in Figure 11 . As shown therein, for entropy_coding_mode_flag = 1, which relates to, for example, context-adaptive binary arithmetic coding or another specific entropy coding mode, the quadtree subdivision of the current coding tree block is signaled within the syntax portion coding_tree via a flag designated split_coding_unit_flag at 400 . As shown in Figure 11 , according to the embodiment described later herein, the root block is subdivided depth-first sequentially as shown in Figure 7A , signaled using the split_coding_unit_flag. Whenever a leaf node is reached, the subdivision represents a coding unit, which is directly encoded using the syntax function coding_unit. This can be seen in FIG11 , when looking at the if-clause at 402 , which checks whether the current split_coding_unit_flag is set. If so, the coding_tree function is repeatedly called, resulting in further split_coding_unit_flags being sent/retrieved at the encoder and decoder, respectively. If not, that is, if split_coding_unit_flag = 0, then the current subblock of the tree root block 200 of FIG7A is a leaf block and to encode this coding unit, the coding_unit function of FIG10 is called at 404 .

In the presently described embodiment, the above-described option is used, according to which merging is only possible for pictures in which the inter-frame prediction mode is available. That is, merging is not used for intra-frame coded slices/pictures. This can be seen from FIG. 12 , where the flag skip_flag is only sent at 406 if the slice type is not equal to the intra-picture slice type, that is, if the current slice to which the current coding unit belongs allows partitions to be inter-frame coded. Merging only concerns, according to this embodiment, prediction parameters related to inter-frame prediction. According to this embodiment, the skip_flag is signaled for the entire coding unit 40 and, if the skip_flag is equal to 1, this flag value is also signaled to the decoder:

1) the current coding unit separation mode is a non-splitting mode, according to which the same is not separated and presents itself as the only separation of the coding unit,

2) The current coding unit/partition is inter-frame coded, i.e., it is specified to inter-frame coding mode,

3) The current code unit/partition accepts merging, and

4) The current coding unit/partition accepts skip mode, ie has skip mode activated.

Thus, if skip_flag is set, the prediction_unit function is called at 408, indicating that the current coding unit is a prediction unit. However, this is not the only possible way to switch to the merge option. Instead, if skip_flag is not set at 406 for the entire coding unit, the prediction type for the non-intra-picture slice coding unit is signaled using the syntax element pred_type at 410, according to which the prediction type is called for any partition of the current coding unit, i.e., at 412, if the current coding unit is not further partitioned. In FIG12, only four different partition options are shown, but the other partition options shown in FIG8 are also available. Another possibility is that the partition option PART_NxN is not available, but the others are. The correlation between the names of the partition modes used in FIG12 and the partition options shown in FIG8 is indicated by the corresponding subscripts under the corresponding partition options in FIG8. Note that the prediction type syntax element pred_type not only signals the prediction mode, i.e., intra-frame or inter-frame coding, but also signals the partition in the case of inter-frame coding mode. The inter-frame coding mode case is discussed further. The function prediction_unit is called for each partition, such as partitions 50 and 60 in the coding order described above. The function prediction_unit begins at 414 by checking the skip_flag. If the skip_flag is set, merge_idx follows at 416. The check at step 414 is to see whether the skip_flag signaled at 406 for the entire coding unit is also set. If not, a merge_flag is again signaled at 418. If the latter is set, a merge_idx follows at 420, indicating the merge candidate for the current partition. Again, merge_flag is signaled for the current partition at 418 only if the current prediction mode of the current coding unit is an inter-frame prediction mode (see 422). That is, if skip_flag is not set, the prediction mode is signaled via pred_type at 410, wherein, for each prediction unit, assuming that pred_type signals that inter-frame coding mode is activated (see 422), a merge-specific flag, namely merge_flag, is correspondingly sent for each following partition if merging is activated for the corresponding partition via the merge index merge_idx.

As can be seen from FIG. 13B , when the prediction parameters for the current prediction unit are sent at 424 , according to this embodiment, merging is only performed if it is not used for the current prediction unit, i.e. because merging is not activated with the skip_flag nor with the corresponding merge_flag of the corresponding partition.

As already indicated above, skip_flag=1 simultaneously signals that no residual data is sent. This can be derived from the fact that for the current coding unit, the residual data transmission at 426 in FIG. 12 only occurs if skip_flag is equal to 0, and can also be derived from the fact that the skip_flag state is checked in real time after its transmission within the else option of the if-clause 428.

Thus far, the embodiments of Figures 11 to 13A and 13B have been described solely under the assumption that entropy_coding_mode_flag is equal to 1. However, the embodiments of Figures 11 to 13A and 13B also encompass embodiments described above with entropy_coding_mode_flag = 0, in which case another entropy coding mode is used for entropy coding the syntax element, such as variable length coding and, for greater precision, context-adaptive variable length coding. For example, the possibility of simultaneously signaling merge activation on the one hand and skip mode on the other hand exists in the embodiments described above, according to which only one of more than two sets of states for the corresponding syntax element is signaled jointly. This will now be described in more detail. However, it should be noted that the possibility of switching between the two entropy coding modes is optional, and therefore, different embodiments can be easily derived from Figures 11 to 13 by allowing only one of the two entropy coding modes.

For example, see Figure 11. If entropy_coding_mode_flag is equal to 0 and the slice_type syntax element signals that the current tree root block belongs to an inter-frame coded slice, that is, the inter-frame coding mode is available, then a syntax element cu_split_pred_part_mode is sent at 430, and this syntax element signals, as indicated by its name, further sub-partitioning information of the current coding unit, activation or deactivation of the skip mode, activation or deactivation of the merge and prediction mode, and corresponding partitioning information. See Listing 1:

List 1

Table 1 indicates the importance of the possible states of the syntax element cu_split_pred_part_mode for the case where the current coding unit is not the smallest size in the quadtree subdivision of the current tree root block. These possible states are listed in the outermost left row of Table 1. Because Table 1 indicates the case where the current coding unit is not the smallest size, there is a state cu_split_pred_part_mode, namely, state 0, which signals that the current coding unit is not an actual coding unit, but must be further subdivided into four units, which are then traversed in depth-first order, again as described in calling the function coding_tree at 432. That is, cu_split_pred_part_mode = 0 signals that the current quadtree subdivision unit of the current tree root block is to be further subdivided into four smaller units, i.e., split_coding_unit_flag = 1. However, if cu_split_pred_part_mode is set to any other possible state, split_coding_unit_flag = 0 and the current unit forms a leaf block of the current tree root block, i.e., a coding unit. In this case, one of the remaining possible states of cu_split_pred_part_mode represents the previously described jointly signaled state, which simultaneously signals that the current coding unit is subject to merging and has skip mode enabled, as indicated by skip_flag being equal to 1 in the third row of Table 1, and simultaneously signals that no further splitting of the current coding unit occurs, i.e., PART_2Nx2N is selected as the split mode. cu_split_pred_part_mode also has a possible state in which merging is enabled and skip mode is not enabled. This is possible state 2, corresponding to skip_flag = 0 and merge_flag = 1, with no split mode active, i.e., PART_2Nx2N. That is, in this case, merge_flag is signaled in advance rather than within the prediction_unit syntax. In the remaining possible states of cu_split_pred_part_mode, inter prediction modes with other partition modes are signaled that split the current coding unit into more than one partition.

List 2

Table 2 shows the significance or syntax of the possible states of cu_split_pred_part_mode for the case where the current coding unit has the smallest possible size according to the quadtree subdivision of the current root block. In this case, according to split_coding_unit_flag = 0, all possible states of cu_split_pred_part_mode correspond to no further subdivision. However, possible state 0 signals skip_flag = 1, i.e., simultaneously signals that merging is enabled and skip mode is in effect. In addition, the same also signals that no splitting occurs, i.e., the partitioning mode PART_2Nx2N. Possible state 1 corresponds to possible state 2 of Table 1, and the same applies to possible state 2 of Table 2, which corresponds to possible state 3 of Table 1.

While the above description of the embodiments of Figures 11 through 13A and 13B has explained most of the functionality and syntax, some further information is presented below.

[0065] When skip_flag[x0][y0] is equal to 1, it specifies that for the current coding unit (see 40 in the accompanying drawings), no further syntax elements are parsed after skip_flag[x0][y0] except for the motion vector predictor index (merge_idx) when decoding a P or B slice. When skip_flag[x0][y0] is equal to 0, it specifies that the coding unit is not skipped. The array indices x0, y0 indicate the position (x0, y0) of the top-left luma sample of the considered coding unit relative to the top-left luma sample of the picture (20 in the accompanying drawings).

When skip_flag[x0][y0] is not present, it will be inferred to be equal to 0.

As mentioned above, if skip_flag[x0][y0] is equal to 1,

- The prediction mode is inferred to be equal to MODE_SKIP

- The partitioning pattern is inferred to be equal to PART_2Nx2N

cu_split_pred_part_mode[x0][y0] specifies the split_coding_unit_flag and when the coding unit is not to be split, skip_flag[x0][y0], merge_flag[x0][y0], the prediction mode and the partitioning mode of the coding unit. The array index x0, y0 indicates the position (x0, y0) of the top-left luma sample of the considered coding unit relative to the top-left luma sample of the picture.

merge_flag[x0][y0] indicates whether the inter-prediction parameters for the current prediction unit (50 and 60 in the figure, i.e., partitions within coding unit 40) are inferred from the neighboring inter-prediction partitions. The array indices x0, y0 indicate the position (x0, y0) of the top-left luma sample of the considered prediction block relative to the top-left luma sample of the picture.

merge_idx[x0][y0] indicates the merge candidate index of the merge candidate list, where x0, y0 indicates the position (x0, y0) of the top-left luma sample of the considered prediction block relative to the top-left luma sample of the picture.

Although not explicitly indicated in the description of Figures 11-13 above, in this embodiment, the determination of the merge candidate or the merge candidate list is exemplified not only using the coding parameters or prediction parameters of spatially adjacent prediction units/partitions, but also using the prediction parameters of temporally adjacent temporally adjacent partitions and previously coded images when forming the candidate list. In addition, combinations of prediction parameters of spatially and/or temporally adjacent prediction units/partitions are used and included in the merge candidate list. Of course, only a subset of these may be used. In particular, Figure 14 shows one possibility for determining spatially adjacent, i.e., spatially adjacent partitions or prediction units. 14 exemplarily shows a prediction unit or partition 60 and pixels _B0 to _B2 and _A0 and _A1 located directly adjacent to the boundary 500 of the partition 60, that is, _B2 is the top left pixel diagonally adjacent to the partition 60, _B1 is the top right pixel located vertically above and adjacent to the partition 60, _B0 is the top right pixel located diagonally to the partition 60, _A1 is the horizontal left of the partition 60 and adjacent to the bottom left pixel, and _A0 is the bottom left pixel located diagonally to the partition 60. A partition containing at least one pixel of _B0 to _B2 and _A0 and _A1 forms a spatial neighborhood and its prediction parameters form a merge candidate.

In order to perform the above selection removing those candidates which would result in another partitioning pattern in which the candidate is also available, the following function can be used:

In particular, candidate N, i.e., the coding/prediction parameters recovered from the prediction unit/partition containing covered pixels N=( _B0 , _B1 , _B2 , _A0 , _A1 ), i.e., position (xN, yN), is removed from the candidate list if any of the following conditions is true (see FIG8 for the partition mode (PartMode) and the corresponding partition index (PartIdx) for retrieving the corresponding partition within the coding unit):

- the partition mode of the current prediction unit is PART_2NxN and the partition index is equal to 1 and the prediction units covering the luma position (xP, yP-1) (PartIdx=0) and the luma position (xN, yN) (Cand.N) have the same motion parameters:

mvLX[xP, yP–1]==mvLX[xN, yN]

refIdxLX[xP, yP–1]==refIdxLX[xN, yN]

predFlagLX[xP, yP–1]==predFlagLX[xN, yN]

- the current prediction unit partition mode is PART_Nx2N and PartIdx is equal to 1 and the prediction units covering luma position (xP−1, yP) (PartIdx=0) and luma (xN, yN) (Cand.N) have the same motion parameters:

mvLX[xP–1,yP]＝＝mvLX[xN,yN]

refIdxLX[xP–1, yP]==refIdxLX[xN, yN]

predFlagLX[xP–1, yP]==predFlagLX[xN, yN]

- The partitioning mode of the current prediction unit is PART_NxN and PartIdx is equal to 3 and the prediction units covering the luma position (xP-1, yP) (PartIdx=2) and the luma position (xP-1, yP-1) (PartIdx=0) have the same motion parameters:

mvLX[xP–1,yP]＝＝mvLX[xP–1,yP–1]

refIdxLX[xP–1, yP]==refIdxLX[xP–1, yP–1]

predFlagLX[xP–1, yP]==predFlagLX[xP–1, yP–1]

And the prediction units including the luma position (xP, yP-1) (PartIdx=1) and the luma position (xN, yN) (Cand.N) have the same motion parameters:

mvLX[xP, yP–1]==mvLX[xN, yN]

refIdxLX[xP, yP–1]==refIdxLX[xN, yN]

predFlagLX[xP, yP–1]==predFlagLX[xN, yN]

The partition mode of the current prediction unit is PART_NxN and PartIdx is equal to 3 and the prediction units covering the luma position (xP, yP-1) (PartIdx=1) and the luma position (xP-1, yP-1) (PartIdx=0) have the same motion parameters:

mvLX[xP, yP–1]==mvLX[xP–1, yP–1]

refIdxLX[xP, yP–1]==refIdxLX[xP–1, yP–1]

predFlagLX[xP, yP–1]==predFlagLX[xP–1, yP–1]

And the same motion parameters are used for the prediction units covering the luma position (xP-1, yP) (PartIdx=2) and the luma position (xN, yN) (Cand.N):

mvLX[xP–1,yP]＝＝mvLX[xN,yN]

refIdxLX[xP–1, yP]==refIdxLX[xN, yN]

In this regard, it should be noted that the position (xP, yP) indicates the topmost pixel of the current partition/prediction unit. That is, according to the first item, all coding parameter candidates are checked, which are derived by directly adopting the corresponding coding parameters of the neighboring prediction unit, namely prediction unit N. However, other coding parameter candidates can be checked in the same way as to whether they are identical to the coding parameters of the corresponding prediction unit, with which merging would result in another partition pattern also supported by the syntax. According to the embodiment described just above, the equality of the coding parameters includes checking the equality of the motion vector, namely mvLX, the reference index, namely refIxLX, and the prediction flag predFlagLX, which indicates whether the parameters associated with reference list X, namely the motion vector and the reference index, with X being 0 or 1, are used for inter-frame prediction.

Please note that the above mentioned possibility of removing coding parameter candidates for adjacent prediction units/partitions also applies to the case of supported asymmetric partition modes, as shown in the right half of FIG8 . In this case, the mode PART_2NxN can represent all horizontal sub-partition modes and PART_Nx2N can correspond to all vertical sub-partition modes. Furthermore, the mode PART_NxN can be excluded from the supported partition modes or partition patterns and in this case, only the first two removal checks will be performed.

With regard to the embodiment of Figures 11-14, it should also be noted that it is possible to exclude intra-frame prediction partitions from the candidate list, ie their coding parameters are, of course, not included in the candidate list.

Furthermore, it should be noted that three sets of contexts can be used for skip_flag, merge_flag, and merge_idx, respectively.

Although some aspects are described in the context of an apparatus, it should be understood that these aspects also represent descriptions of corresponding methods, where a block or device corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent descriptions of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps can be performed by (or using) a hardware device, such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some or more of the most important method steps can be performed using such a device.

Depending on the implementation requirements, embodiments of the present invention may be implemented in hardware or software. The implementation may be performed using a digital storage medium having electronically readable control signals stored thereon, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM, or flash memory, which cooperates (or is capable of cooperating) with a programmable computer system to enable the respective methods to be performed. Thus, the digital storage medium can be read by a computer.

Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

Generally, embodiments of the present invention can be implemented as a computer program product having a program code, which, when executed on a computer, is operable to perform the operations of one of the methods. The program code can, for example, be stored on a machine-readable carrier.

Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the inventive method is, therefore, a computer program comprising a program code for performing one of the methods described herein, when the computer program runs on a computer.

A further embodiment of the method according to the invention is, therefore, a data carrier (or a digital storage medium or a computer-readable medium) comprising, recorded thereon, a computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recording medium is generally tangible and/or non-transitory.

A further embodiment of the method according to the invention is therefore a data stream or a sequence of signals representing the computer program for carrying out one of the methods described herein. This data stream or signal sequence can, for example, be configured to be transmitted via a data communication connection (e.g. via the Internet).

A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured or adapted to perform one of the methods described herein.

A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

A further embodiment according to the present invention comprises an apparatus or a system configured to transmit (e.g., electronically or optically) a computer program for performing the methods described herein to a receiver. The receiver is, for example, a computer, a mobile device, a memory device, or the like. The apparatus or system comprises, for example, a file server for transmitting the computer program to the receiver.

In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can operate in conjunction with a microprocessor to perform one of the methods described herein. In general, these methods are preferably performed using any hardware device.

The embodiments described above are merely illustrative of the principles of the present invention. Those skilled in the art will appreciate that the present invention and the details described herein are susceptible to various modifications and variations. Therefore, the present invention is limited only by the scope of the appended claims and is not limited to the specific details described and illustrated by the embodiments herein.

Claims (35)

1. An apparatus for decoding a bitstream, wherein an image (20) is encoded into the bitstream, the apparatus comprising: A sub-segmenter (82) is configured to sub-segment the image (20) into a plurality of sample sets (40); A merger (84) is configured to merge the plurality of sample sets (40) into a plurality of groups, each having one or more sample sets; A decoder (86) is configured to decode the image (20) using encoding parameters transmitted in the bitstream in units of the groups of the sample set, wherein the decoder (86) is configured to decode the image for a predetermined sample set by predicting the image (20), decoding a prediction residual for the predetermined sample set, and combining the prediction residual with a prediction generated due to the prediction of the image (20); An extractor (88) is configured to extract the prediction residual and the encoding parameters from the bitstream (30) together with one or more syntax elements for each of at least one subset of the sample set (40), the one or more syntax elements indicating whether the corresponding sample set (40) will be merged into one of the plurality of groups along with another sample set (40), wherein the merger (84) is configured to perform the merge in response to the one or more syntax elements. The extractor and the merger are configured to sequentially step into the plurality of sample sets according to the scanning order of a sample set, and for a current sample set, the extractor and the merger are configured as follows: Using the first context, arithmetic decoding is performed to extract the first binary syntax element from the bitstream; If the first binary syntax element is determined to be a first binary state, the current sample set is merged into one of the multiple groups by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with one of the multiple groups, the extraction of the prediction residual for the current sample set is skipped, and the sample set is stepped to the next sample set in the sample set scanning order. If the first binary syntax element is determined to be a second binary state, then the second syntax element is extracted from the bitstream using arithmetic decoding with a second context different from the first context; and Depending on the second syntax element, Using a third context different from the first and second contexts, arithmetic decoding is employed to extract a third syntax element from the bitstream. The current sample set is then merged into one of the plurality of groups by inferring that the encoding parameters indicated by the third syntax element for the current sample set are equal to the encoding parameters associated with one of the sets of predetermined candidate sample sets adjacent to the corresponding sample set. Simultaneously, at least one additional syntax element related to the prediction residual for the current sample set is extracted. Extract the encoding parameters for the current sample set. Simultaneously, at least one additional syntax element related to the prediction residuals used for the current sample set is extracted. The extractor is further configured to extract sub-segmentation information from the bitstream using entropy decoding, and the sub-segmenter is configured to sub-segment the image into multiple sample sets in response to the sub-segmentation information. 2. The apparatus of claim 1, wherein the sub-segmenter is configured to sub-segment the image into a plurality of sample sets in response to the sub-segmentation information using multi-tree sub-segmentation. 3. The apparatus of claim 1, wherein the bitstream further includes a depth map associated with the image. 4. The apparatus of claim 1, wherein the information sample array is one of sample arrays relating to different planes of the image, the different planes being encoded independently of each other. 5. An apparatus for encoding an image, the apparatus comprising: A sub-segmenter (72) is configured to sub-segment the image into a plurality of sample sets; A merger (74) is configured to merge the plurality of sample sets into multiple groups, each of which has one or more sample sets; An encoder (76) is configured to encode the image using encoding parameters transmitted in a bitstream in units of groups of the sample set, wherein the encoder (76) is configured to encode the image by predicting the image and encoding the prediction residual for a predetermined sample set; A bitstream generator (78) is configured to insert the prediction residuals and the encoding parameters, along with one or more syntax elements for each of at least a subset of the plurality of sample sets, into a bitstream, the one or more syntax elements indicating whether the corresponding sample set is merged into one of the plurality of groups along with another sample set. The bitstream generator (78) is configured to sequentially step into the plurality of sample sets according to a sample set scanning order, and for a current sample set, the bitstream generator (78) is configured to: Using arithmetic encoding with the first context, the first binary syntax element is inserted into the bit stream; Wherein, when determined to be the first binary state, the first binary syntax element indicates that the current sample set is merged into one of the multiple groups by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with one of the multiple groups, and indicates that the extraction of the prediction residual for the current sample set is skipped and the sample set is stepped to the next sample set in the sample set scanning order; If the first binary syntax element is determined to be the second binary state. Using a second context different from the first context, the second syntax element is inserted into the bitstream using arithmetic encoding; and Insert at least one additional syntax element relating to the prediction residuals used for the current sample set. The second syntax element indicates whether to merge the current sample set into one of the plurality of groups or whether to perform the extraction of the encoding parameters for the current sample set. The inserter is configured to, if the second syntax element indicates merging the current sample set, arithmetically decode using a third context different from the first and second contexts, insert a third syntax element into the bitstream, and merge the current sample set by inferring the encoding parameters indicated by the third syntax element that are associated with one of the predetermined candidate sample sets adjacent to the corresponding sample set, whose encoding parameters for the current sample set are equal to those of one of the predetermined candidate sample sets. The bitstream generator is further configured to insert sub-segmentation information into the bitstream using entropy coding, wherein the sub-segmenter indicates how to sub-segment the image into multiple sample sets. 6. The apparatus of claim 5, wherein the bitstream further includes a depth map associated with the image. 7. The apparatus of claim 5, wherein the information sample array is one of sample arrays relating to different planes of the image, the different planes being encoded independently of each other. 8. A method for decoding a bitstream, wherein an image (20) is encoded into the bitstream, the method comprising: The image (20) is subdivided into multiple sample sets (40); The multiple sample sets (40) are merged into multiple groups, each of which has one or more sample sets; The image (20) is decoded using encoding parameters transmitted in the bitstream in units of the groups of the sample set, wherein the decoder (86) is configured to decode the image (20) for a predetermined sample set by predicting the image (20), decoding a prediction residual for the predetermined sample set, and combining the prediction residual with a prediction generated due to the prediction of the image (20). Together with one or more syntax elements for each of at least one subset of the sample set (40), the prediction residual and the encoding parameters are extracted from the bitstream (30), the one or more syntax elements indicating whether the corresponding sample set (40) will be merged into one of the plurality of groups along with another sample set (40), wherein the merger (84) is configured to perform the merge in response to the one or more syntax elements. The method includes sequentially stepping through the plurality of sample sets according to the scanning order of a sample set, and, for a current sample set, Using the first context, arithmetic decoding is performed to extract the first binary syntax element from the bitstream; If the first binary syntax element is determined to be a first binary state, the current sample set is merged into one of the multiple groups by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with one of the multiple groups, the extraction of the prediction residual for the current sample set is skipped, and the sample set is stepped to the next sample set in the sample set scanning order. If the first binary syntax element is determined to be a second binary state, then the second syntax element is extracted from the bitstream using arithmetic decoding with a second context different from the first context; and Depending on the second syntax element, Using a third context different from the first and second contexts, arithmetic decoding is employed to extract a third syntax element from the bitstream. The current sample set is then merged into one of the plurality of groups by inferring that the encoding parameters indicated by the third syntax element for the current sample set are equal to the encoding parameters associated with one of the sets of predetermined candidate sample sets adjacent to the corresponding sample set. Simultaneously, at least one additional syntax element related to the prediction residual for the current sample set is extracted. Extract the encoding parameters for the current sample set. Simultaneously, at least one additional syntax element related to the prediction residuals used for the current sample set is extracted. Specifically, entropy decoding is used to extract sub-segmentation information from the bitstream, and the image is sub-segmented into multiple sample sets in response to the sub-segmentation information. 9. The method of claim 8, wherein the bitstream further includes a depth map associated with the image. 10. The method of claim 8, wherein the information sample array is one of sample arrays associated with different planes of the image, the different planes being encoded independently of each other. 11. A method for encoding an image, the method comprising: The image is subdivided into multiple sample sets; The multiple sample sets are merged into multiple groups, each of which has one or more sample sets; The image is encoded using encoding parameters transmitted in a bitstream in units of groups of the sample set, wherein the encoder (76) is configured to encode the image by predicting the image and encoding the prediction residual for a predetermined sample set; The prediction residuals and the encoding parameters are inserted into a bitstream along with one or more syntax elements for each of at least a subset of the plurality of sample sets, the one or more syntax elements indicating whether the corresponding sample set is merged into one of the plurality of groups along with another sample set. The method includes sequentially stepping through the plurality of sample sets according to the scanning order of a sample set, and, for a current sample set, Using arithmetic encoding with the first context, the first binary syntax element is inserted into the bit stream; Wherein, when determined to be the first binary state, the first binary syntax element indicates that the current sample set is merged into one of the multiple groups by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with one of the multiple groups, and indicates that the extraction of the prediction residual for the current sample set is skipped and the sample set is stepped to the next sample set in the sample set scanning order; If the first binary syntax element is determined to be the second binary state. Using a second context different from the first context, the second syntax element is inserted into the bitstream using arithmetic encoding; and Insert at least one additional syntax element relating to the prediction residuals used for the current sample set. The second syntax element indicates whether to merge the current sample set into one of the plurality of groups or whether to perform the extraction of the encoding parameters for the current sample set. The inserter is configured to, if the second syntax element indicates merging the current sample set, arithmetically decode using a third context different from the first and second contexts, insert a third syntax element into the bitstream, and merge the current sample set by inferring the encoding parameters indicated by the third syntax element that are associated with one of the predetermined candidate sample sets adjacent to the corresponding sample set, whose encoding parameters for the current sample set are equal to those of one of the predetermined candidate sample sets. Specifically, entropy coding is used to insert sub-segmentation information into the bitstream, whereby the sub-segmentation information indicates how the image is sub-segmented into multiple sample sets. 12. The method of claim 11, wherein the bitstream further includes a depth map associated with the image. 13. The method of claim 11, wherein the information sample array is one of sample arrays associated with different planes of the image, the different planes being encoded independently of each other. 14. A method for decoding a bitstream, wherein the method comprises: Receive and decode a bitstream encoded by the following: The image is subdivided into multiple sample sets; The multiple sample sets are merged into multiple groups, each of which has one or more sample sets; The image is encoded using encoding parameters transmitted in the bitstream in units of the sample set, wherein the encoder (76) is configured to encode the image by predicting the image and encoding the prediction residual for a predetermined sample set; The prediction residuals and the encoding parameters are inserted into a bitstream along with one or more syntax elements for each of at least a subset of the plurality of sample sets, the one or more syntax elements indicating whether the corresponding sample set is merged into one of the plurality of groups along with another sample set. The method includes sequentially stepping through the plurality of sample sets according to the scanning order of a sample set, and, for a current sample set, Using arithmetic encoding with the first context, the first binary syntax element is inserted into the bit stream; Wherein, when determined to be the first binary state, the first binary syntax element indicates that the current sample set is merged into one of the multiple groups by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with one of the multiple groups, and indicates that the extraction of the prediction residual for the current sample set is skipped and the sample set is stepped to the next sample set in the sample set scanning order; If the first binary syntax element is determined to be the second binary state. Using a second context different from the first context, the second syntax element is inserted into the bitstream using arithmetic encoding; and Insert at least one additional syntax element relating to the prediction residuals used for the current sample set. The second syntax element indicates whether to merge the current sample set into one of the plurality of groups or whether to perform the extraction of the encoding parameters for the current sample set. The inserter is configured to, if the second syntax element indicates merging the current sample set, arithmetically decode using a third context different from the first and second contexts, insert a third syntax element into the bitstream, and merge the current sample set by inferring the encoding parameters indicated by the third syntax element that are associated with one of the predetermined candidate sample sets adjacent to the corresponding sample set, whose encoding parameters for the current sample set are equal to those of one of the predetermined candidate sample sets. Specifically, entropy coding is used to insert sub-segmentation information into the bitstream, whereby the sub-segmentation information indicates how the image is sub-segmented into multiple sample sets. 15. The method of claim 14, wherein the bitstream further comprises a depth map associated with the image. 16. The method of claim 14, wherein the information sample array is one of sample arrays associated with different planes of the image, the different planes being encoded independently of each other. 17. A method for decoding a bitstream, wherein the method comprises: Receive and decode a bitstream, the bitstream comprising: Predicting residuals and coding parameters, wherein the coding parameters are transmitted in the bitstream in units of groups of sample sets of an image, wherein the groups are generated by subsegmenting the image into multiple sample sets and merging the multiple sample sets into multiple groups of sample sets; One or more syntactic elements for each of at least one subset of the sample sets, indicating whether the corresponding sample set will be merged into one of the plurality of groups along with another sample set. The method includes sequentially stepping through the plurality of sample sets according to the scanning order of a sample set, and, for a current sample set, Using arithmetic encoding with the first context, the first binary syntax element is inserted into the bit stream; Wherein, when determined to be the first binary state, the first binary syntax element indicates that the current sample set is merged into one of the multiple groups by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with one of the multiple groups, and indicates that the extraction of the prediction residual for the current sample set is skipped and the sample set is stepped to the next sample set in the sample set scanning order; If the first binary syntax element is determined to be the second binary state. Using a second context different from the first context, the second syntax element is inserted into the bitstream using arithmetic encoding; and Insert at least one additional syntax element relating to the prediction residuals used for the current sample set. The second syntax element indicates whether to merge the current sample set into one of the plurality of groups or whether to perform the extraction of the encoding parameters for the current sample set. The inserter is configured to, if the second syntax element indicates merging the current sample set, arithmetically decode using a third context different from the first and second contexts, insert a third syntax element into the bitstream, and merge the current sample set by inferring the encoding parameters indicated by the third syntax element that are associated with one of the predetermined candidate sample sets adjacent to the corresponding sample set, whose encoding parameters for the current sample set are equal to those of one of the predetermined candidate sample sets. Specifically, entropy coding is used to insert sub-segmentation information into the bitstream, whereby the sub-segmentation information indicates how the image is sub-segmented into multiple sample sets. 18. The method of claim 17, wherein the bitstream further comprises a depth map associated with the image. 19. The method of claim 17, wherein the information sample array is one of sample arrays associated with different planes of the image, the different planes being encoded independently of each other. 20. A method for storing an image, comprising: A bitstream is stored on a digital storage medium, the bitstream comprising: Predicting residuals and coding parameters, wherein the coding parameters are transmitted in the bitstream in units of groups of sample sets of an image, wherein the groups are generated by subsegmenting the image into multiple sample sets and merging the multiple sample sets into multiple groups of sample sets; One or more syntactic elements for each of at least one subset of the sample sets, indicating whether the corresponding sample set will be merged into one of the plurality of groups along with another sample set. The method includes sequentially stepping through the plurality of sample sets according to the scanning order of a sample set, and, for a current sample set, Using arithmetic encoding with the first context, the first binary syntax element is inserted into the bit stream; Wherein, when determined to be the first binary state, the first binary syntax element indicates that the current sample set is merged into one of the multiple groups by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with one of the multiple groups, and indicates that the extraction of the prediction residual for the current sample set is skipped and the sample set is stepped to the next sample set in the sample set scanning order; If the first binary syntax element is determined to be the second binary state. Using a second context different from the first context, the second syntax element is inserted into the bitstream using arithmetic encoding; and Insert at least one additional syntax element relating to the prediction residuals used for the current sample set. The second syntax element indicates whether to merge the current sample set into one of the plurality of groups or whether to perform the extraction of the encoding parameters for the current sample set. The inserter is configured to, if the second syntax element indicates merging the current sample set, arithmetically decode using a third context different from the first and second contexts, insert a third syntax element into the bitstream, and merge the current sample set by inferring the encoding parameters indicated by the third syntax element that are associated with one of the predetermined candidate sample sets adjacent to the corresponding sample set, whose encoding parameters for the current sample set are equal to those of one of the predetermined candidate sample sets. Specifically, entropy coding is used to insert sub-segmentation information into the bitstream, whereby the sub-segmentation information indicates how the image is sub-segmented into multiple sample sets. 21. The method of claim 20, wherein the bitstream further includes a depth map associated with the image. 22. The method of claim 20, wherein the information sample array is one of sample arrays associated with different planes of the image, the different planes being encoded independently of each other. 23. A method for transmitting an image, comprising: Transmitting a bit stream over a transmission medium, the bit stream comprising: Predicting residuals and coding parameters, wherein the coding parameters are transmitted in the bitstream in units of groups of sample sets of an image, wherein the groups are generated by subsegmenting the image into multiple sample sets and merging the multiple sample sets into multiple groups of sample sets; One or more syntactic elements for each of at least one subset of the sample sets, indicating whether the corresponding sample set will be merged into one of the plurality of groups along with another sample set. The method includes sequentially stepping through the plurality of sample sets according to the scanning order of a sample set, and, for a current sample set, Using arithmetic encoding with the first context, the first binary syntax element is inserted into the bit stream; Wherein, when determined to be the first binary state, the first binary syntax element indicates that the current sample set is merged into one of the multiple groups by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with one of the multiple groups, and indicates that the extraction of the prediction residual for the current sample set is skipped and the sample set is stepped to the next sample set in the sample set scanning order; If the first binary syntax element is determined to be the second binary state. Using a second context different from the first context, the second syntax element is inserted into the bitstream using arithmetic encoding; and Insert at least one additional syntax element relating to the prediction residuals used for the current sample set. The second syntax element indicates whether to merge the current sample set into one of the plurality of groups or whether to perform the extraction of the encoding parameters for the current sample set. The inserter is configured to, if the second syntax element indicates merging the current sample set, arithmetically decode using a third context different from the first and second contexts, insert a third syntax element into the bitstream, and merge the current sample set by inferring the encoding parameters indicated by the third syntax element that are associated with one of the predetermined candidate sample sets adjacent to the corresponding sample set, whose encoding parameters for the current sample set are equal to those of one of the predetermined candidate sample sets. Specifically, entropy coding is used to insert sub-segmentation information into the bitstream, whereby the sub-segmentation information indicates how the image is sub-segmented into multiple sample sets. 24. The method of claim 23, wherein the bitstream further includes a depth map associated with the image. 25. The method of claim 23, wherein the information sample array is one of sample arrays associated with different planes of the image, the different planes being encoded independently of each other. 26. An apparatus configured to decode a bitstream (30) to which an image (20) is encoded, the image being divided into multiple blocks, and the bitstream indicating whether the corresponding blocks of each block are encoded in an intra-image prediction mode or an inter-image prediction mode, the apparatus being configured to: For a current block, extract the tag from the bit stream (30). Use the aforementioned marker to determine: Whether the encoding parameters associated with the current block are set based on a merge candidate through merging or obtained from the bitstream (30), A second decision is made regarding whether the current block (40) of the image (20) is reconstructed based solely on a prediction signal dependent on the encoding parameters associated with the current block (40) without any residual data, or by refining the prediction signal dependent on the encoding parameters associated with the current block (40) using residual data within the bitstream (30). Whether to encode the current block using an inter-image prediction mode, If the marker in the bitstream (30) indicates that the encoding parameters associated with the current block (40) are set according to a merge candidate, the residual data of the current block is not transmitted, and the current block is encoded in an inter-image prediction mode, Then a merge is performed, such that the coding parameters associated with the current block (40) are obtained by setting coding parameters according to a merge candidate, and the current block (40) of the image (20) is reconstructed using an inter-image prediction mode based solely on a prediction signal dependent on the coding parameters without any residual data; and If the marker in the bitstream (30) indicates that the encoding parameters associated with the current block (40) are not set according to a merge candidate, Syntax elements are then extracted from the bitstream, indicating whether the current block is encoded using intra-image prediction mode or inter-image prediction mode. If the syntax elements indicate that the current block is encoded using inter-image prediction mode, then for each segment of the current block, the current block is further segmented, and a merge marker indicates the actuation of the individual merging of the corresponding segments. The apparatus is configured to extract sub-segmentation information from the bitstream using entropy decoding, and the apparatus is configured to sub-segment the image into a plurality of blocks in response to the sub-segmentation information. 27. The apparatus of claim 26, wherein the apparatus is configured such that the apparatus If the marker within the bitstream does not indicate that the encoding parameters associated with the current block (40) are set according to a merge candidate, Then, for each segment of the current block, in response to the merge marker within the bitstream, the device depends on the merge marker. The merging of the corresponding segments of the current block is initiated, such that the encoding parameters associated with the corresponding segments of the current block (40) are obtained by setting encoding parameters according to a merge candidate, residual data of the corresponding segments of the current block (40) is obtained from the bitstream (30), and the corresponding segments of the current block (40) are reconstructed based on the prediction signal and the residual data, or Extract the encoding parameters associated with the corresponding segment of the current block (40) from the bit stream (30), obtain residual data for the corresponding segment of the current block (40) from the bit stream (30), and reconstruct the corresponding segment of the current block (40) based on the prediction signal and the residual data. 28. An apparatus for decoding a bitstream to which an image (20) is encoded, the apparatus comprising: A sub-segmenter (82) is configured to sub-segment the image (20) into multiple blocks (40) of samples; A merger (84) is configured to merge blocks (40) into multiple groups of one or more blocks accordingly; A decoder (86) is configured to decode the image (20) using encoding parameters transmitted in the bitstream in units of the blocks, wherein the decoder (86) is configured to decode the image for a predetermined block by predicting the image (20), decoding a prediction residual for the predetermined block, and combining the prediction residual with a prediction generated from predicting the image (20), wherein the bitstream indicates whether the corresponding blocks of each block are encoded in an intra-image prediction mode or an inter-image prediction mode; An extractor (88) is configured to extract the prediction residual and the encoding parameters from the bitstream (30) together with one or more syntax elements for each of at least a subset of the blocks (40), the one or more syntax elements indicating whether the corresponding block (40) is encoded in an inter-image prediction mode, whether the corresponding block (40) will be merged into one of the plurality of groups along with another block (40), and whether residual data of the corresponding block (40) is transmitted, wherein the merger (84) is configured to perform merging in response to the one or more syntax elements. Wherein, one of the possible states of the one or more syntax elements indicates that the corresponding block (40) is encoded in an inter-image prediction mode, the corresponding block (40) will be merged into one of the plurality of groups along with another block (40), and the corresponding block does not have a prediction residual that is encoded and inserted into the bitstream (30), wherein the extractor (88) is configured to, for each block in which one or more syntax elements does not determine one of the possible states, extract from the bitstream a syntax element indicating whether the corresponding block is encoded in an intra-image prediction mode or an inter-image prediction mode, wherein the separation information indicates the separation of the corresponding block, and if the corresponding block is encoded in an inter-image prediction mode, a merge flag is set for each separation of the corresponding block according to the separation information, the merge flag individually actuating the merging of the corresponding separation. The extractor is further configured to extract sub-segmentation information from the bitstream using entropy decoding, and the sub-segmenter is configured to sub-segment the image into multiple blocks in response to the sub-segmentation information. 29. The apparatus of claim 28, wherein the sub-segmenter is configured to sub-segment the image into multiple blocks in response to the sub-segmentation information using multi-tree sub-segmentation. 30. An apparatus for encoding an image (20) into a bitstream (30), the sample array of the image being divided into multiple blocks (40), such that the bitstream indicates whether the corresponding blocks of each block are encoded in an intra-image prediction mode or an inter-image prediction mode, the apparatus being configured to encode a marker into the bitstream for a current block, the marker indicating: Whether the encoding parameters associated with the current block (40) are set by merging based on a merge candidate or obtained from the bitstream (30), Is the current block (40) of the image (20) reconstructed based solely on a prediction signal that depends on the encoding parameters associated with the current block (40) without any residual data, or is the reconstruction based on the prediction signal that depends on the encoding parameters associated with the current block (40) refined from the residual data within the bitstream (30)? Whether to encode the current block using an inter-image prediction mode, If the marker in the bitstream (30) indicates that the encoding parameters associated with the current block (40) are set according to a merge candidate, the residual data of the current block is not transmitted, and the current block is encoded in an inter-image prediction mode, The encoding parameters associated with the current block (40) are then set according to the merge candidate, and the current block is encoded using an inter-image prediction mode, such that the current block (40) of the image (20) is reconstructed using an inter-image prediction mode based solely on a prediction signal dependent on the encoding parameters without any residual data; and If the marker in the bitstream (30) indicates that the encoding parameters associated with the current block (40) are not set according to a merge candidate, Then, a syntax element is inserted into the bitstream, indicating whether the current block is encoded using intra-image prediction mode or inter-image prediction mode. Furthermore, if the syntax element indicates that the current block is encoded using inter-image prediction mode, the current block is further divided for each segment, and a merge marker indicates the actuation of the individual merging of the corresponding segments. The device is configured to insert sub-segmentation information into the bitstream using entropy coding, the sub-segmentation information indicating how the image is subdivided into multiple blocks. 31. An apparatus for encoding an image into a bitstream, comprising: A sub-segmenter (72) is configured to sub-segment the image (20) into multiple blocks of samples; A merger (74) is configured to merge blocks (40) into multiple groups, each containing one or more blocks; An encoder (76) is configured to encode the image using encoding parameters transmitted in the bitstream in units of the group of blocks, wherein the encoder (76) is configured to encode the image by predicting the image and encoding a prediction residual for a predetermined block, wherein the bitstream indicates whether the corresponding block of each block is encoded in an intra-image prediction mode or an inter-image prediction mode. A bitstream generator (78) is configured to insert the prediction residual and the encoding parameters together with one or more syntax elements for each of at least a subset of the blocks into a bitstream, the one or more syntax elements indicating whether the corresponding block is encoded in an inter-image prediction mode, whether the corresponding block will be merged into one of the plurality of groups along with another block, and whether residual data of the corresponding block is not transmitted. Wherein, one of the possible states of the one or more syntax elements indicates that the corresponding block is encoded in an inter-image prediction mode, the corresponding block will be merged into one of the plurality of groups along with another block, and the corresponding block does not have a prediction residual that is encoded and inserted into the bitstream, wherein the bitstream generator (78) is configured to insert into the bitstream for each block for which one or more syntax elements do not determine one of the possible states, wherein the syntax element indicates whether the corresponding block is encoded in an intra-image prediction mode or an inter-image prediction mode, wherein the separation information in the bitstream indicates the separation of the corresponding block, and for each separation of the corresponding block according to the separation information, a merge flag is set, the merge flag individually actuating the merging of the corresponding separation. The bitstream generator is further configured to insert sub-segmentation information into the bitstream using entropy coding, the sub-segmentation information indicating how the image is subdivided into multiple blocks. 32. A method for decoding a bitstream (30) to which an image (20) is encoded, the image being divided into multiple blocks, and the bitstream indicating whether the corresponding blocks of each block are encoded in an intra-image prediction mode or an inter-image prediction mode, the method comprising: For a current block, extract the tag from the bit stream (30). Use the aforementioned marker to determine: Whether the encoding parameters associated with the current block are set based on a merge candidate through merging or obtained from the bitstream (30), Is the current block (40) of the image (20) reconstructed based solely on a prediction signal that depends on the encoding parameters associated with the current block (40) without any residual data, or is the reconstruction based on the prediction signal that depends on the encoding parameters associated with the current block (40) refined from the residual data within the bitstream (30)? Whether to encode the current block using an inter-image prediction mode, If the marker in the bitstream (30) indicates that the encoding parameters associated with the current block (40) are set according to a merge candidate, the residual data of the current block is not transmitted, and the current block is encoded in an inter-image prediction mode, Then a merge is performed, such that the encoding parameters associated with the current block (40) are obtained by setting encoding parameters according to a merge candidate, and The current block (40) of the image (20) is reconstructed using an inter-image prediction mode based solely on a prediction signal dependent on the coding parameters, without any residual data; and If the marker in the bitstream (30) indicates that the encoding parameters associated with the current block (40) are not set according to a merge candidate, Syntax elements are then extracted from the bitstream, indicating whether the current block is encoded using intra-image prediction mode or inter-image prediction mode. If the syntax elements indicate that the current block is encoded using inter-image prediction mode, the current block is further divided for each segment, and a merge marker indicates the actuation of merging the corresponding segments. Specifically, entropy decoding is used to extract sub-segmentation information from the bitstream, and the image is sub-segmented into multiple blocks in response to the sub-segmentation information. 33. A method for decoding a bitstream to which an image (20) is encoded, the method comprising: The image (20) is subdivided into multiple blocks (40); Accordingly, blocks (40) are merged into multiple groups of one or more blocks; The image (20) is decoded using encoding parameters transmitted in the bitstream in units of the blocks, and the image is decoded for a predetermined block by predicting the image (20), decoding a prediction residual for the predetermined block, and combining the prediction residual with a prediction generated from predicting the image (20), wherein the bitstream indicates whether the corresponding blocks of each block are encoded in an intra-image prediction mode or an inter-image prediction mode. Together with one or more syntax elements for each of at least a subset of the blocks (40), the prediction residuals and the encoding parameters are extracted from the bitstream (30), the one or more syntax elements indicating whether the corresponding block (40) is encoded in an inter-image prediction mode, whether the corresponding block (40) will be merged into one of the plurality of groups along with another block (40), and whether residual data of the corresponding block (40) is transmitted, wherein merging is performed in response to the one or more syntax elements. Wherein, one of the possible states of the one or more syntax elements indicates that the corresponding block (40) is encoded in an inter-image prediction mode, the corresponding block (40) will be merged into one of the plurality of groups along with another block (40), and the corresponding block does not have a prediction residual that is encoded and inserted into the bitstream (30), wherein the extraction includes, for each block in which one or more syntax elements does not determine one of the possible states, extracting from the bitstream a syntax element indicating whether the corresponding block is encoded in an intra-image prediction mode or an inter-image prediction mode, wherein the separation information indicates the separation of the corresponding block, and if the corresponding block is encoded in an inter-image prediction mode, a merge marker is set for each separation of the corresponding block according to the separation information, the merge marker individually actuating the merging of the corresponding separation. Specifically, entropy coding is used to insert sub-segmentation information into the bitstream, whereby the sub-segmentation information indicates how the image is subdivided into multiple blocks. 34. A method for encoding an image (20) into a bitstream (30), the sample array of the image being divided into multiple blocks (40), such that the bitstream indicates whether the corresponding blocks of each block are encoded in an intra-image prediction mode or an inter-image prediction mode, the method comprising: For a given block, a tag is encoded into the bitstream, the tag indicating: Whether the encoding parameters associated with the current block (40) are set by merging based on a merge candidate or obtained from the bitstream (30), Regarding the current block (40) of the image (20), is it reconstructed based solely on a prediction signal that depends on the encoding parameters associated with the current block (40) without any residual data, or is it reconstructed by refining the prediction signal that depends on the encoding parameters associated with the current block (40) using residual data within the bitstream (30)? Whether to encode the current block using an inter-image prediction mode, If the marker in the bitstream (30) indicates that the encoding parameters associated with the current block (40) are set according to a merge candidate, the residual data of the current block is not transmitted, and the current block is encoded in an inter-image prediction mode, The encoding parameters associated with the current block (40) are then set according to the merge candidate, and the current block is encoded using an inter-image prediction mode, such that the current block (40) of the image (20) is reconstructed using an inter-image prediction mode based solely on a prediction signal dependent on the encoding parameters without any residual data; and If the marker in the bitstream (30) indicates that the encoding parameters associated with the current block (40) are not set according to a merge candidate, Then, a syntax element is inserted into the bitstream, indicating whether the current block is encoded in intra-image prediction mode or inter-image prediction mode, and wherein, if the syntax element indicates that the current block is encoded in inter-image prediction mode, the current block is further divided for each segment, and a merge marker indicates the actuation of the individual merging of the corresponding segments. Specifically, entropy coding is used to insert sub-segmentation information into the bitstream, whereby the sub-segmentation information indicates how the image is subdivided into multiple blocks. 35. A method for encoding an image, the method comprising: The image is subdivided into multiple blocks of samples; Each of the blocks (40) is merged into a group of one or more blocks; The image is encoded using encoding parameters transmitted in a bitstream in units of the blocks, wherein the encoder (76) is configured to encode the image by predicting the image and encoding a prediction residual for a predetermined block, wherein the bitstream indicates whether the corresponding blocks of each block are encoded in an intra-image prediction mode or an inter-image prediction mode. The prediction residual and the encoding parameters are inserted into a bitstream along with one or more syntax elements for each of at least a subset of the blocks, the one or more syntax elements indicating whether the corresponding block is encoded in an inter-image prediction mode, whether the corresponding block will be merged into one of the plurality of groups along with another block, and whether the residual data of the corresponding block is not transmitted. Wherein, one of the possible states of the one or more syntax elements indicates that the corresponding block is encoded in an inter-image prediction mode, the corresponding block will be merged into one of the plurality of groups along with another block, and the corresponding block does not have a prediction residual that is encoded and inserted into the bitstream, wherein the insertion includes, for each block in which one or more syntax elements do not determine one of the possible states, insertion into the bitstream, wherein the syntax element indicates whether the corresponding block is encoded in an intra-image prediction mode or an inter-image prediction mode, wherein the separation information in the bitstream indicates the separation of the corresponding block, and for each separation of the corresponding block according to the separation information, a merge flag is provided, the merge flag individually actuating the merging of the corresponding separation. Specifically, entropy coding is used to insert sub-segmentation information into the bitstream, whereby the sub-segmentation information indicates how the image is subdivided into multiple blocks.