WO2012172659A1 - Image encoding method, image decoding method, image encoding apparatus, and image decoding apparatus - Google Patents

Abstract

The purpose of the present invention is to reduce the amount of calculation necessary in a bidirectional prediction method. An image encoding apparatus of an embodiment is provided with a motion compensation prediction unit, an interpolation image generating unit, and an encoding unit. The motion compensation prediction unit generates a motion compensation prediction image that is predicted from a reference image on the basis of a motion vector. The interpolation image generating unit generates a bidirectional prediction image that is the result of interpolating a plurality of motion compensation prediction images that are generated for a plurality of reference images. The encoding unit generates encoded data that is the result of encoding the difference between an input image and the bidirectional prediction image. When a first set, which indicates a combination of pixel positions of each of the plurality of motion compensation prediction images for obtaining the pixel value of the first pixel position of the bidirectional prediction image, is included among particular sets that are sets of predetermined pixel positions, the interpolation image generating unit adds the pixel values at each of the pixel positions contained in the first set and interpolates the added pixel values to calculate the pixel value of the first pixel position.

Description

Image encoding method, image decoding method, image encoding device, and image decoding device

Embodiments described herein relate generally to an image encoding method, an image decoding method, an image encoding device, and an image decoding device.

In the moving image coding technology, an interpolation image with an accuracy higher than the integer pixel accuracy is created, and the prediction efficiency is improved by using bi-directional prediction in which a prediction value is calculated by an average value or a weighted average of two interpolation images. The method of aiming is common. In the bidirectional prediction method, two unidirectional interpolation images are created, and an average value of the two interpolation images is obtained to create a bidirectional prediction image. For this reason, it is known that a rounding error in determining the average value becomes a problem. There is a technique for reducing a rounding error in the middle of calculation by generating an average after holding a unidirectional interpolation image generation result with a bit length longer than an input pixel bit length when generating a bidirectional prediction interpolation image.

K. Ugur, J. Lainema, A. HALlapuro, “High precision bi-direction AL averaging,” JCTVC-D321, 4th. JCT-VC Meeting, Daegu, KR, 20-28 January, 2011

However, in the conventional technique, since a plurality of unidirectional interpolation images are generated and an average value of the plurality of interpolation images is calculated, an interpolation image generation process twice as usual is required, and the amount of calculation is large. There was a problem of increasing.

The image coding apparatus according to the embodiment includes a motion compensation prediction unit, an interpolation image generation unit, and a coding unit. The motion compensated prediction unit generates a motion compensated predicted image that is predicted based on the motion vector from the reference image. The interpolation image generation unit generates a bidirectional prediction image obtained by interpolating a plurality of motion compensated prediction images generated for a plurality of reference images. The encoding unit generates encoded data obtained by encoding the difference between the input image and the bidirectional prediction image. The interpolated image generation unit is configured such that a first set representing a combination of pixel positions of a plurality of motion compensated prediction images for obtaining a pixel value of the first pixel position of the bidirectional prediction image is a predetermined pixel position. When the pixel values are included in the specific group, the pixel values at the respective pixel positions included in the first group are added, and the pixel value at the first pixel position is calculated by interpolating the added pixel values.

1 is a block diagram of an image encoding device according to a first embodiment. 1 is a block diagram of an image decoding apparatus according to a first embodiment. The block diagram which a prediction image generation part shows. The block diagram of the bidirectional | two-way prediction part concerning 1st Embodiment. The block diagram of the bidirectional | two-way interpolation image generation part of 1st Embodiment. The figure which shows an example of a fraction pixel position. The figure which shows an example of the fraction precision information corresponding to a fraction pixel position. The figure which shows the example of the motion compensation image of integer pixel precision. The figure of the table which shows an example of the switching conditions of a switch part. The block diagram of a 1st image generation part. The block diagram of a 2nd image generation part. The flowchart of the interpolation image generation process in 1st Embodiment. The figure which showed the frequency | count of multiplication per pixel by 1st Embodiment. The figure which showed the multiplication frequency per pixel by the conventional method. The figure which showed the frequency | count of multiplication per pixel by 2nd Embodiment. The figure which showed the frequency | count of multiplication per pixel by 3rd Embodiment. The figure which showed the frequency | count of multiplication per pixel by 4th Embodiment. FIG. 5 is a hardware configuration diagram of an apparatus according to first to fourth embodiments.

Exemplary embodiments of an image encoding method, an image decoding method, an image encoding device, and an image decoding device according to the present invention will be described below in detail with reference to the accompanying drawings.
(First embodiment)
The image encoding device and the image decoding device according to the first embodiment first detect each pixel when a set of pixel positions of two reference images used for bidirectional prediction corresponds to a predetermined set (specific set). After adding the pixel values of the positions, multiplication with the filter coefficient of the filter used for interpolation is executed. Thereby, it is possible to reduce the number of operations when generating the interpolation image.

FIG. 1 is a block diagram illustrating a configuration example of an image encoding device 600 according to the first embodiment. The image coding apparatus 600 according to the first embodiment includes a subtracting unit 602, a transform / quantization unit 603, an inverse quantization / inverse transform unit 604, an entropy coding unit 605, an addition unit 606, and a frame memory. Unit 608, predicted image generation unit 610, motion vector search unit 612, and encoding control unit 614. The image encoding device 600 generates encoded data 615 from the input moving image signal 601.

For example, the input moving image signal 601 is input to the image encoding device 600 in units of frames. The input moving image signal 601 is divided into units such as macro blocks. The subtraction unit 602 outputs a prediction error signal that is a difference between the predicted image signal 611 generated by the predicted image generation unit 610 and the input moving image signal 601.

The transform / quantization unit 603 orthogonally transforms the prediction error signal by, for example, discrete cosine transform (DCT), and executes quantization processing to generate quantized transform coefficient information. The quantized transform coefficient information is bifurcated, and one is input to the entropy encoding unit 605. The other of the bifurcated quantized transform coefficient information is input to the inverse quantization / inverse transform unit 604. The inverse quantization / inverse transform unit 604 performs inverse quantization and inverse transform processing on the quantized transform coefficient information as processing reverse to the processing of the transform / quantization unit 603, and reproduces the prediction error signal. . The adding unit 606 adds the prediction error signal and the prediction image signal. As a result, a decoded image signal 607 is generated.

The decoded image signal 607 is input to the frame memory unit 608. The frame memory unit 608 performs a filtering process or the like on the decoded image signal 607, and then uses the decoded image signal 609 as a reference image signal 609 input to the predicted image generation unit 610 based on the prediction control information 507. It is determined whether 607 is stored.

The reference image signal 609 is input to the predicted image generation unit 610 and also to the motion vector search unit 612. The motion vector search unit 612 generates motion vector information 613 from the input moving image signal 601 and the reference image signal 609. The motion vector information 613 is input to the predicted image generation unit 610 and is also sent to the entropy encoding unit 605. The predicted image generation unit 610 generates a predicted image signal 611 from the reference image signal 609, the prediction control information 507, and the motion vector information 613.

The encoding control unit 614 controls operations of the transform / quantization unit 603, the predicted image generation unit 610, the frame memory unit 608, and the like. The prediction control information 507 generated by the encoding control unit 614 is input to the predicted image generation unit 610 and the frame memory unit 608 and is also sent to the entropy encoding unit 605. The entropy encoding unit 605 includes various codes such as quantized transform coefficient information from the transform / quantization unit 603, prediction control information 507 from the encoding control unit 614, and motion vector information 613 from the motion vector search unit 612. Encoded data is entropy-encoded to generate encoded data according to a predetermined syntax.

FIG. 2 is a block diagram illustrating a configuration example of an image decoding apparatus 700 corresponding to the image encoding apparatus 600. The image decoding apparatus 700 includes an entropy decoding unit 702, an inverse quantization / inverse transform unit 703, an addition unit 704, a frame memory unit 706, and a predicted image generation unit 610. The image decoding device 700 generates a playback video signal 707 from the encoded data 701.

The entropy decoding unit 702 performs entropy decoding processing of the encoded data 701 according to a predetermined syntax. The entropy decoding unit 702 decodes the encoded data 701 to obtain quantized transform coefficient information, prediction control information 711, and motion vector information 712. The decoded quantized transform coefficient information is input to the inverse quantization / inverse transform unit 703. The decoded prediction control information 711 and motion vector information 712 are input to the predicted image generation unit 610.

The inverse quantization / inverse transform unit 703 performs inverse quantization and inverse orthogonal transform processing on the quantized transform coefficient information to reproduce a prediction error signal. The adder 704 adds the prediction error signal and the prediction image signal 710 to generate a decoded image signal 705.

The decoded image signal 705 is input to the frame memory unit 706. The frame memory unit 706 performs a filtering process on the decoded image signal 705 and outputs it as a reproduced moving image signal 707. The frame memory unit 706 determines whether to store the filtered decoded image signal 705 based on the prediction control information 711. The stored decoded image signal 705 is input to the predicted image generation unit 610 as a reference image signal 708.

The predicted image generation unit 610 generates a predicted image signal 710 using the reference image signal 708, the prediction control information 711, and the motion vector information 712.

FIG. 3 is a block diagram illustrating a configuration example of the predicted image generation unit 610 included in the image encoding device 600 and the image decoding device 700. The predicted image generation unit 610 includes a switch 503, a bidirectional prediction unit 504, a unidirectional prediction unit 505, and an intra prediction unit 506. The predicted image generation unit 610 generates a predicted image signal 509 from the reference image signal 502, the prediction control information 507, and the motion vector information 508.

The prediction control information 507 includes, for example, information specifying which of the bidirectional prediction unit 504, the unidirectional prediction unit 505, and the intra prediction unit 506 is used. The switch 503 refers to this information and switches so as to select any one of the bidirectional prediction unit 504, the unidirectional prediction unit 505, and the intra prediction unit 506.

The reference image signal 502 is input to any of the bidirectional prediction unit 504, the unidirectional prediction unit 505, and the intra prediction unit 506 selected by the switch 503. When the bidirectional prediction unit 504 is selected, the bidirectional prediction unit 504 generates a motion compensated image signal using the reference image signals 502 and the motion vector information 508 from a plurality of reference frames, and performs bidirectional prediction. Based on this, a predicted image signal 509 is generated. When the unidirectional predictor 505 is selected, the unidirectional predictor 505 generates a motion compensated image signal using the reference image signal 502 and the motion vector information 508 from a single reference frame, and generates a predicted image. A signal 509 is generated. When the intra prediction unit 506 is selected, the intra prediction unit 506 generates a predicted image signal 509 using the reference image signal 502 in the screen.

FIG. 4 is a block diagram illustrating a configuration example of the bidirectional prediction unit 504 according to the first embodiment. The bidirectional prediction unit 504 includes a motion compensation prediction unit 107, an accuracy calculation unit 104, and a bidirectional interpolation image generation unit 109. The bidirectional prediction unit 504 receives the reference image signal 102 and the motion vector information 103 as inputs, and outputs a bidirectional prediction image signal 110.

The motion compensation prediction unit 107 refers to the reference image signal with integer pixel accuracy based on the integer accuracy information 105 from the accuracy calculation unit 104. Here, H. According to H.264 / AVC, the two reference images are distinguished by being referred to as a reference image in list 0 and a reference image in list 1. In the present embodiment, when the motion vector 103 is set to 1/4 pixel accuracy, the accuracy calculation unit 104 calculates an integer from the motion vector (mvLX [0], mvLX [1]) of the list L (L is 0 or 1). The accuracy information (xInt, yInt) and the fractional accuracy information (xFrac, yFrac) are calculated by the following equations.
xInt = xP + (mvLX [0] >> 2) + x
yInt = yP + (mvLX [1] >> 2) + y
xFrac = mvLX [0] & 3
yFrac = mvLX [1] & 3

The fractional accuracy of the motion vector may be other fractional accuracy such as 1/8 pixel accuracy or 1/16 pixel accuracy in addition to the 1/4 pixel accuracy, and the same argument holds. Here, the upper left reference pixel of the motion compensation block is (xP, yP), and the pixel position in the block is represented by (x, y).

The integer accuracy information (xInt, yInt) of list 0 and list 1 is sent to the motion compensation prediction unit 107. The motion compensation prediction unit 107 reads out pixel data in a necessary range from the reference image signals of the list 0 and the list 1, and sends the pixel data to the bidirectional interpolation image generation unit 109.

The bidirectional interpolation image generation unit 109 performs bidirectional prediction based on the fractional accuracy information (xFrac, yFrac) of each of list 0 and list 1 and the motion compensated prediction image 108 with integer pixel accuracy from the motion compensation prediction unit 107. An image signal 110 is generated and output.

FIG. 5 is a block diagram illustrating a configuration example of the bidirectional interpolation image generation unit 109 of the present embodiment. The bidirectional interpolation image generation unit 109 includes a switch unit 201, a first image generation unit 202, and a second image generation unit 203. The bidirectionally interpolated image generation unit 109 performs bidirectional operation based on the motion compensated predicted image 108 and the fractional accuracy information 106 by either the first image generation unit 202 or the second image generation unit 203 switched by the switch unit 201. A predicted image signal 110 is generated.

Here, a specific example of the fractional accuracy information of the bidirectional prediction unit 504 in the case of 1/4 pixel accuracy will be described. FIG. 6 is a diagram illustrating an example of the fractional pixel position. FIG. 7 is a diagram illustrating an example of fraction accuracy information corresponding to the fractional pixel position.

As shown in FIG. 6, a, b, c, d, e, f, g, h, i, j, k, n, p, q, and r represent fractional pixel positions. 7, xFrac represents the fractional pixel accuracy information in the horizontal direction of the list L, and yFrac represents the fractional pixel accuracy in the vertical direction of the list L. Here, A corresponding to xFrac = 0 and yFrac = 0 represents an integer pixel position in the list L.

FIG. 8 is a diagram illustrating an example of a motion compensated image with integer pixel accuracy referred to by the motion compensation prediction unit 107 in the case of 1/4 pixel accuracy. A _{L, x, y} indicates pixel values of integer pixel positions x, y in the list L. The bidirectional prediction unit 504 generates an interpolated image between integer pixels with reference to the integer pixel positions 0 and 0.

In the present embodiment, a two-dimensional interpolation image with 1/4 pixel accuracy is created. When the filter coefficient of the FIR filter is F, the following relationship is established. The same argument can be made with 1/8 pixel accuracy and 1/16 pixel accuracy. The moving image signal to be applied can be applied not only to a luminance signal but also to a color difference signal. For example, in the case of 4: 2: 0 format, if the luminance is 1/4 pixel accuracy, the color difference signal has 1/8 pixel accuracy, and the method of this embodiment can be applied independently or simultaneously. In the case of 4: 2: 2 format, if the luminance is 1/4 pixel accuracy, the color difference signal has 1/4 pixel accuracy in the vertical direction and 1/8 pixel accuracy in the horizontal direction, and the same argument can be made.

Here, it can be seen that the value of the filter coefficient for 1/2 pixel accuracy is symmetric with respect to the interpolation target pixel position. From this, it is possible to multiply together pixel positions existing at positions symmetrical to the interpolation target pixel position of the motion compensated prediction image. The filter coefficient for 1/4 pixel accuracy and the filter coefficient for 3/4 pixel accuracy are obtained by inverting the filter coefficient. Therefore, when the pixel accuracy is 3/4, the pixel accuracy is the same as 1/4 by inverting the pixel position of the input motion compensated prediction image around the pixel position having the pixel accuracy of 1/2. It turns out that it becomes processing.

Specifically, as an example of a so-called 8-tap filter with N = 4, a filter coefficient such as the following Expression 2 is shown. The variable shift0 representing the accuracy of the filter coefficient is 6.

_{TL, Z} is the pixel position Z of the list L (Z is any one of A, a, b, c, d, e, f, g, h, i, j, k, n, p, q, r) Interpolated image of bipredSample _{predSampleL0 and predSampleL1} are the bi-predictive image signals 110 in the case of the combination of the pixel position predSampleL0 in list 0 and the pixel position predSampleL1 in list 1. In addition, the following variables are defined. Bit Depth represents the pixel bit length of the input moving image signal 601. Here, “<<” and “>>” are operations representing a left arithmetic shift and a right arithmetic shift, respectively. When shifting with a negative value, the shift direction is inverted to be an arithmetic shift with a positive value. That is, for example, “x >> (− 2)” is set to “x << 2”. A ternary operator used in C language or the like is used. And define the following variables:
shift1 = BitDepth-8-S
shift2 = BitDepth-2-S
shift3 = 14-BitDepth + S
offset0 =
(Shift0> 0)? (1 << (shift0-1)): 0;
offset1 =
(Shift1> 0)? (1 << (shift1-1)): 0;
offset2 =
(Shift2> 0)? (1 << (shift2-1)): 0;
offset3 =
(Shift3> 0)? (1 << (shift3-1)): 0;

Shift1, shift2, and shift3 are variables for controlling calculation accuracy, and S is a parameter. When S is 0, all interpolation processes can be accommodated in 16-bit precision. When S is Bit Depth-2, there is no loss of accuracy due to rounding processing in the middle. offset 0, offset 1, offset 2, offset 3 are offset values for rounding. Note that the value of S may be different between the first image generation unit 202 and the second image generation unit 203.

The bidirectional prediction image signal predSample is finally obtained by the following equation (1).
predSample = clip ((bipleSample _{predSampleL0, predSampleL1} + (offset3 + 1)) >> (shift3 + 1)) (1)

Here, the function clip is a function that limits the value to 0 or more and (1 << BitDepth) −1 or less.

FIG. 9 is a table showing an example of switching conditions of the switch unit 201. This table includes pixel positions (A, a, b, c, d, e, f, g, h, i, j, k, n, p, q indicated by xFrac and yFrac in list 0 and list 1, respectively. , R), which one of the first image generation unit 202 and the second image generation unit 203 is to be selected is shown. Since this is an example of 1/4 pixel accuracy, there are 256 combinations of pixel positions in list 0 and list 1. The first row represents the pixel position of list 0, and the first column represents the pixel position of list 1. In FIG. 9, when the value is 0, the switch unit 201 switches so as to select the first image generation unit 202, and when the value is 1, the switch unit 201 switches so as to select the second image generation unit 203. An example is shown.

FIG. 10 is a block diagram illustrating a configuration example of the first image generation unit 202. The first image generation unit 202 includes a list 0 interpolation unit 301, a list 1 interpolation unit 302, and an addition unit 303. The first image generation unit 202 uses the motion compensated prediction image 108 and the fractional accuracy information 106 to create the interpolated images of list 0 and list 1, respectively, and then calculates an average value thereof by the adding unit 303. Thus, the bidirectional prediction image signal 110 is created.

The list 0 interpolation unit 301 generates an interpolated image by performing an interpolation process on the reference image of list 0. The list 1 interpolation unit 302 generates an interpolation image by performing an interpolation process on the reference image of list 1. For example, the list 0 interpolation unit 301 and the list 1 interpolation unit 302 generate an interpolation image by calculating an interpolation value obtained by interpolating the pixel values of the pixels of the reference image of list 0 and the reference image of list 1, respectively. The adding unit 303 adds the interpolation image of list 0 and the interpolation image of list 1.

　なお、Ｔ_Ｌ，jの右辺の式は、Ｓ＝０の時に符号付き１５ビット整数の範囲を超えるこの可能性があるため、ある一定の範囲の値に制限してもよい。例えば、下限が－２¹⁵で上限が２¹⁵－１の範囲に結果を制限する。 In the following, a specific calculation method will be described using equations for how to obtain bipredSample _{predSampleL0 and predSampleL1} in the first image generation unit 202. The interpolated image of list 0 is T _{0 and predSampleL0,} and the interpolated image of list 1 is T _{1 and predSampleL1} . bipredSample _{predSampleL0 and predSampleL1} are calculated as shown in the following equations.
bipredSample _{predSampleL0, predSampleL1} =
T _{0, predSampleL0} + T _{1, predSampleL1}
T _{L, A} = A _{L, 0,0} << shift3

Note that the expression on the right side of _{TL, j} may exceed the range of the signed 15-bit integer when S = 0, and therefore may be limited to a certain range of values. For example, limit the results to a range with a lower limit of −2 ¹⁵ and an upper limit of 2 ¹⁵ −1.

FIG. 11 is a block diagram illustrating a configuration example of the second image generation unit 203. The second image generation unit 203 includes a replacement unit 401, an addition unit 402, and an interpolation unit 403. The second image generation unit 203 replaces the pixel position of the motion compensated prediction image 108 as necessary from the motion compensated prediction image 108 and the fractional accuracy information 106 (replacement unit 401), and performs addition processing (addition unit 402). ), The bidirectional prediction image signal 110 is generated by performing the interpolation process (interpolation unit 403).

When the combination of the pixel positions corresponds to a predetermined group as a combination for replacing the pixel position, the replacement unit 401 replaces the pixel position included in the group with a predetermined pixel position. For example, in the case of a combination of pixel positions with a pixel accuracy of 1/4 and 3/4, the replacement unit 401 inverts a pixel position with a pixel accuracy of 3/4 around a pixel position with a pixel accuracy of 1/2. In the case of a configuration in which the pixel position is not replaced, the replacement unit 401 may not be provided. For example, when only the combination of pixel positions having the same filter coefficient is the processing target of the second image generation unit 203, it is not necessary to include the replacement unit 401 because it is not necessary to invert the pixel positions.

The addition unit 402 adds the pixel values at each pixel position included in the combination of pixel positions after replacement as necessary. The interpolation unit 403 creates the bidirectional prediction image signal 110 by performing interpolation processing using a filter on the image obtained by the addition.

Note that the configuration including the replacement unit 401, the addition unit 402, and the interpolation unit 403 is merely an example, and any other configuration may be used as long as the bidirectional prediction image signal 110 can be calculated using Equations 6 to 33 described later. Good.

Unlike the first image generation unit 202, the second image generation unit 203 realizes a reduction in the number of computations by performing interpolation processing after adding the motion compensated prediction image 108 first.

Hereinafter, a specific calculation method in which the second image generation unit 203 calculates bipred _{Sample pred} Sample _{L0 and pred} Sample _L1 based on the fractional accuracy information will be described.

As described above, for example, in the case of a combination of pixel positions having a value of “1” in the table of FIG. 9, bipleSample _{predSampleL0 and predSampleL1} are calculated by the second image generation unit 203. Each of the following equations 6 to 33 represents an example of a calculation formula of bipredSample _{predSampleL0 and predSampleL1} for each combination of pixel positions having a value of “1” in the table of FIG.

In these examples, when filtering is performed both in the horizontal direction and in the vertical direction, if the number of computations does not change, the vertical direction is performed first and the horizontal direction is performed later, but vice versa. Even if it exists, it is realizable similarly.

In the case of performing the weighted prediction, in the case of the first image generation unit 202, H. 264 / AVC Similarly, using a weight coefficient _{W L} and the offset coefficient _{O L} and scaling LS list L with respect to the interpolation image _{predSample = clip ((W 0 ×} T 0, predSampleL0 + W 1 × T 1, predSampleL1 + ( offset3 + LS + 2)) >> (shift3 + LS + 2) + ((O ₀ + O ₁ +1) >> 1))
A weighted prediction image is created. _{Alternatively} , a weighted interpolation image TW _{L, predSampleLx} represented by the following equation (2) is created, and the following equation (3) is substituted into equation (1) to create a weighted prediction image. .
TW _{L, predSampleLx} =
_{(W L × T L, predSampleLx} + (1 << (LS-1))) >> LS + O L ··· (2)
bipredSample _{predSampleL0, predSampleL1} = (TW _{0, pred Sample L0} + TW _{1, pred Sample L1} ) (3)

In the case of the second image generation unit 203, for the integer pixels A _{x, y} ,
AW _{L, x, y} = W _L × A _{L, x, y}
And weighted integer pixel values AWL _{, x, y} are generated, and then the images bipleSample _{predSampleL0 and predSampleL1} generated by the second image generation unit 203 are
predSample = clip ((bipleSample _{predSampleL0, predSampleL1} + (offset3 + LS + 2)) >> (shift3 + LS + 2) + ((O ₀ + O ₁ +1) >> 1)
Thus, a weighted prediction image is created. Or, for integer pixel A in advance,
AW _{L, x, y} = (W _L × A _{L, x, y} + (1 << (LS-1))) >> LS + _OL
The weighted integer pixel values AWL _{, x, y} are generated, and the image bipleSample _{predSampleL0 and predSampleL1} generated by the second image generation unit 203 are substituted into the formula (1) to obtain a weighted prediction image Create

Next, interpolation image generation processing by the image encoding device 600 according to the first embodiment will be described with reference to FIG. FIG. 12 is a flowchart showing an overall flow of the interpolation image generation processing in the first embodiment.

The motion compensation prediction unit 107 reads out image signals with integer pixel accuracy from the reference image signals of list 0 and list 1 according to the integer accuracy information calculated by the accuracy calculation unit 104 (step S101). The motion compensation prediction unit 107 outputs the read image signal to the bidirectional interpolation image generation unit 109 as a motion compensated prediction image 108 with integer pixel accuracy.

The bidirectional interpolation image generation unit 109 inputs the fractional accuracy information (xFrac, yFrac) of the list 0 and the list 1 calculated by the accuracy calculation unit 104 (step S102). The bidirectionally interpolated image generation unit 109 determines whether to use the second image generation unit 203 based on the combination of pixel positions corresponding to the fractional accuracy information of the list 0 and the list 1 (step S103).

For example, the bidirectional interpolated image generation unit 109 obtains pixel positions corresponding to the fractional accuracy information for each of the list 0 and the list 1. When the fractional accuracy information (xFrac, yFrac) = (1, 1), the bidirectionally interpolated image generation unit 109 can specify that the pixel position is “e” as shown in FIG. Next, the bidirectionally interpolated image generation unit 109 determines whether or not the combination of the pixel position obtained for the list 0 and the pixel position obtained for the list 1 is a predetermined combination (specific set). . For example, the bidirectional interpolation image generation unit 109 performs the above determination with a combination of pixel positions having a value of “1” in the table of FIG. Then, when the combination of the pixel position obtained for list 0 and the pixel position obtained for list 1 is a specific set, bi-directionally interpolated image generation unit 109 determines to use second image generation unit 203. If it is not a specific group, the bidirectional interpolation image generation unit 109 determines to use the first image generation unit 202.

When the first image generation unit 202 is used (step S103: No), the list 0 interpolation unit 301 of the first image generation unit 202 creates an interpolation image of list 0, and the list 1 interpolation unit of the first image generation unit 202 302 creates an interpolated image of list 1 (step S104). Next, the adding unit 303 adds the interpolated image of list 0 and the interpolated image of list 1 to create and output a bidirectional prediction image signal 110 (step S105).

When the second image generation unit 203 is used (step S103: Yes), the second image generation unit 203 creates the bidirectional prediction image signal 110 according to the equations 6 to 33 (steps S106 to S107).

First, when the combination of pixel positions corresponds to a predetermined group as a combination for replacing a pixel position, the replacement unit 401 replaces the pixel position included in the group with a predetermined pixel position. The adding unit 402 adds the pixel value of the motion compensated predicted image 108 for list 0 and the pixel value of the motion compensated predicted image for list 1 (step S106). The interpolation unit 403 performs an interpolation process using a filter on the image obtained by the addition, and generates and outputs the bidirectional prediction image signal 110 (step S107).

As described above, in this embodiment, when the combination of the pixel positions of the two reference images satisfies a specific condition, the filter processing can be executed after adding the pixel values. Therefore, compared with the conventional method of creating an average value after creating an interpolation image independently in list 0 and list 1, this embodiment is for limiting the number of multiplications and the calculation accuracy in filter processing. The number of operations such as clipping processing can be reduced. That is, the bidirectionally interpolated image generation process can be efficiently realized.

FIG. 13 is a diagram showing the number of multiplications per pixel according to the first embodiment when N = 4. FIG. 14 is a diagram showing the number of multiplications per pixel by a conventional method (a method in which all the interpolated pixels are calculated by the first image generation unit 202) when N = 4.

As shown in FIG. 13, when this embodiment is applied, the maximum number of multiplications is 104, and the average number of multiplications is 51.92. On the other hand, as shown in FIG. 14, when the conventional method is applied, the maximum number of multiplications is 144, and the average number of multiplications is 62.5.

As described above, in the image encoding device according to the first embodiment, when the set of pixel positions of each of the two reference images used for bidirectional prediction corresponds to a specific set, the pixel value of each pixel position is first determined. After the addition, multiplication with the filter coefficient is executed. Thereby, the calculation amount of bidirectional prediction can be reduced.

The combination of fractional accuracy information to which the second image generation unit 203 is applied may be a part of the above combination. In other words, for example, a part of the expressions 6 to 33 may be used. Even in this case, the amount of calculation can be reduced at least for the combination to which the second image generation unit 203 is applied.

(Second Embodiment)
The image encoding device and the image decoding device according to the second embodiment further reduce the amount of calculation by the first image generation unit. The first image generation unit of the second embodiment uses a filter with a short (small) tap length in the case of a pixel position with a large calculation amount.

For example, in the first image generation unit 202 of the first embodiment, the combination of fraction accuracy information of list 0 and list 1 is a combination of f, i, k, q and e, g, p, r. 32 are the worst multiplications (144 times, see FIG. 14). Therefore, the first image generation unit of the second embodiment uses a filter with a short tap length that satisfies N ′ <N when calculating an interpolation image at pixel positions e, g, p, and r. The following Expression 34 is an expression for calculating the interpolated images T ′ _{L, e} , T ′ _{L, g} , T ′ _{L, p} , T ′ _{L, r} at the pixel positions of e, g, p, r in the list L. To express.

FIG. 15 is a diagram showing the number of multiplications per pixel according to the second embodiment. In the case of N = 4 and N ′ = 3, the portion that required the worst 104 multiplications becomes 78 multiplications. The average of 256 patterns is 48.67, and the number of multiplications can be further reduced by about 3 times from the first embodiment. In the present embodiment, 32 types of combinations of f, i, k, q and e, g, p, r are applied, but a part of these or another combination may be used. For example, in addition to this, for example, eight combinations of j, which is the number of multiplications of 92 times in FIG. 15, and e, g, p, r may be included. Alternatively, N ′ may be applied to all or part of the combinations of e, f, g, i, k, p, q, and r in list 0 and list 1, which have a large number of operations. .

(Third embodiment)
The image encoding device and the image decoding device according to the third embodiment perform the second image generation by approximating the list 0 and the list 1 to have the same fractional accuracy in the case of a pixel position having a large calculation amount. It is possible to generate a bidirectional prediction image by the unit 203.

As described above, the worst multiplication is performed when 32 combinations of the fractional accuracy information of list 0 and list 1 are combinations of f, i, k, q and e, g, p, r. It becomes the number of times. Therefore, in the third embodiment, for example, xFrac = 1/2 and yFrac = 1/4 where the list 0 is the position of f, and xFrac = 1/4, yFrac = 1/4 where the list 1 is the position of e. In the case of the combination of 4, both are approximated to xFrac = 3/8 and yFrac = 1/4, respectively. In this case, a filter represented by the following formula for newly generating an interpolation image with 3/8 pixel accuracy is introduced.
F _{3/8 , x} = F _{5/8, −x + 1} (x = −N + 1,..., N)

Specifically, prediction formulas shown in the following formulas 35 to 37 are introduced for 32 combinations of f, i, k, q and e, g, p, r.

FIG. 16 is a diagram showing the number of multiplications per pixel according to the third embodiment. By introducing the above prediction formula, in the case of N = 4, the portion where the worst 104 times of multiplication has been necessary so far becomes 72, 40 or 36 times. The average of 256 is 45.80 times, which can be further reduced by about 6 times from the first embodiment. In this embodiment, 32 combinations of f, i, k, q and e, g, p, r are applied, but some or a combination of these may be used. For example, in addition to this, eight types of combinations of j, which is the number of multiplications of 92 times in FIG. 16, and e, g, p, r may be included.

(Fourth embodiment)
The image encoding device and the image decoding device according to the fourth embodiment generate a second image by replacing list 0 and list 1 with pixel positions having the same fractional accuracy in the case of pixel positions with a large amount of calculation. It is possible to generate a bidirectional prediction image by the unit 203.

Specifically, bipred Sample _{e, k} to be calculated by the first image generation unit 202
bipredSample _{k, e}
bipredSample _{g, i}
bipredSample _{i, g}
bipred Sample _{e, q}
bipred Sample _{q, e}
bipredSample _{f, p}
bipred Sample _{p, f}
bipred Sample _{f, r}
bipredSample _{r, f}
bipred Sample _{g, q}
bipred Sample _{q, g}
bipredSample _{i, r}
bipredSample _{r, i}
bipredSample _{k, p}
bipred Sample _{p, k}
Are replaced with bipreted Sample _{j, j} in the case of the second image generation unit 203.

In addition, bipred Sample _{e, f} to be calculated by the first image generation unit 202
bipred Sample _{f, e}
bipred Sample _{e, i}
bipredSample _{i, e}
Are replaced with bipreted Sample _{e, e} in the case of the second image generation unit 203.

In addition, bipred Sample _{f, g} to be calculated by the first image generation unit 202
bipred Sample _{g, f}
bipredSample _{g, k}
bipredSample _{k, g}
Are replaced with bipred Sample _{g, g} in the case of the second image generation unit 203.

In addition, bipredSample _{i, p} to be calculated by the first image generation unit 202
bipred Sample _{p, i}
bipred Sample _{p, q}
bipred Sample _{q, p}
Are replaced with bisampleSample _{p, p} in the case of the second image generation unit 203.

In addition, bipred Sample _{k, r} to be calculated by the first image generation unit 202
bipredSample _{r, k}
bipred Sample _{q, r}
bipred Sample _{r, q}
Are replaced with bisampled _{r, r} in the case of the second image generation unit 203.

FIG. 17 is a diagram showing the number of multiplications per pixel according to the fourth embodiment. In this embodiment, in the case of N = 4, the portion where the worst 96 times of multiplication is necessary is 16 or 64 times. The average of 256 ways is about 44.80 times, which can be further reduced about 7 times from the first embodiment.

In this embodiment, 32 combinations of f, i, k, q and e, g, p, r are applied, but some or a combination of these may be used. For example, in addition to this, in the case of a combination of 92 times of multiplication and e, g, p, and r in FIG. 17, all eight of these may be replaced with bipledSample _{j, j.} . Alternatively, among these eight types _, combinations including e, g, p, and r may be replaced with bisampled sample _{e, e} , bipreded sample _{g, g} , bipreded sample _{p, p} , bipreded sample _{r, r} , respectively.

As described above, according to the first to fourth embodiments, the number of operations when generating an interpolation image can be reduced.

Next, the hardware configuration of the devices (image encoding device and image decoding device) according to the first to fourth embodiments will be described with reference to FIG. FIG. 18 is an explanatory diagram showing the hardware configuration of the devices according to the first to fourth embodiments.

The devices according to the first to fourth embodiments are connected to a control device such as a CPU (CentrAL Processing Unit) 51, a storage device such as a ROM (Read Only Memory) 52 and a RAM (Random Access Memory) 53, and a network. A communication I / F 54 that performs communication and a bus 61 that connects each unit are provided.

The program executed by the devices according to the first to fourth embodiments is provided by being incorporated in advance in the ROM 52 or the like.

The program executed by the apparatus according to the first to fourth embodiments is a file in an installable or executable format and is a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R. (Compact Disk Recordable), DVD (DigitAL Versatile Disk), etc. may be recorded on a computer-readable recording medium and provided as a computer program product.

Furthermore, the program executed by the apparatuses according to the first to fourth embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. . The program executed by the devices according to the first to fourth embodiments may be provided or distributed via a network such as the Internet.

The program executed by the devices according to the first to fourth embodiments can cause the computer to function as each unit (motion compensation prediction unit, bidirectional interpolation image generation unit, accuracy calculation unit, etc.) of the above-described device. In this computer, the CPU 51 can read a program from a computer-readable storage medium onto a main storage device and execute the program.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

104 Accuracy calculation unit 106 Fractional accuracy information 107 Motion compensation prediction unit 108 Motion compensated prediction image 109 Bidirectional interpolation image generation unit 110 Bidirectional prediction image signal 201 Switch unit 202 First image generation unit 203 Second image generation unit 301 List 0 interpolation Unit 302 list 1 interpolation unit 303 addition unit 401 replacement unit 402 addition unit 403 interpolation unit 502 reference image signal 503 switch 504 bidirectional prediction unit 505 unidirectional prediction unit 506 intra prediction unit 507 prediction control information 508 motion vector information 509 prediction image signal 600 Image coding apparatus 601 Input video signal 602 Subtraction unit 603 Transform / quantization unit 604 Inverse quantization / inverse transform unit 605 Entropy coding unit 606 Addition unit 607 Decoded image signal 608 Frame memory unit 609 Reference image signal 610 Predicted image Generation unit 611 predicted image No. 612 Motion vector search unit 613 Motion vector information 614 Encoding control unit 615 Encoded data 700 Image decoding device 701 Encoded data 702 Entropy decoding unit 703 Inverse quantization / inverse transform unit 704 Adder 705 Decoded image signal 706 Frame memory Unit 707 playback video signal 708 reference image signal 710 prediction image signal 711 prediction control information 712 motion vector information

Claims (18)

A motion compensated prediction step for generating a motion compensated prediction image predicted based on a motion vector from a reference image;
An interpolated image generation step of generating a bi-directional prediction image obtained by interpolating the plurality of motion compensation prediction images generated for the plurality of reference images;
An encoding step for generating encoded data obtained by encoding the difference between the input image and the bidirectional prediction image,
In the interpolation image generation step, a first set representing a combination of pixel positions of a plurality of motion compensated prediction images for obtaining a pixel value of a first pixel position of the bidirectional prediction image is a predetermined pixel. When included in a specific set which is a set of positions, the pixel value of each pixel position included in the first set is added, and the pixel value of the first pixel position is calculated by interpolating the added pixel value ,
An image encoding method characterized by the above. When the first set is not included in the specific set, the interpolation image generation step calculates an interpolation value obtained by interpolating the pixel value for each pixel value of each pixel position included in the first set, and the interpolation Calculating a pixel value of the first pixel position by adding the values;
The image encoding method according to claim 1, wherein: In the interpolation image generation step, when the first set is not included in the specific set, the tap length of at least some of the filters used for calculating the interpolation value is made smaller than other filters.
The image encoding method according to claim 2. In the interpolation image generation step, when the first set is not included in the specific set, the first set is replaced with one of the specific sets determined in advance according to the first set. Calculating the pixel value of the first pixel position by adding the pixel values of each pixel position included in the specific set and interpolating the added pixel values;
The image encoding method according to claim 2. The specific set is a set of pixel positions having the same fractional precision;
The image encoding method according to claim 1, wherein: The specific set is a set of pixel positions whose fractional accuracy is symmetric with respect to 1/2,
In the interpolation image generation step, when the first set is included in the specific set, one of the pixel positions included in the first set is inverted with respect to the first pixel position. Adding the pixel value and the pixel value of the other pixel position among the pixel positions included in the first set, and calculating the pixel value of the first pixel position by interpolating the added pixel value;
The image encoding method according to claim 1, wherein: The specific set is a set of pixel positions whose fractional accuracy is ½,
In the interpolation image generation step, when the first set is included in the specific set, the pixel value of each pixel position included in the first set and the same filter coefficient as each pixel position included in the first set Calculating the pixel value at the first pixel position by interpolating the pixel value at the pixel position using
The image encoding method according to claim 1, wherein: The specific set is a set of pixel positions with different fractional accuracy,
In the interpolation image generation step, when the first set is included in the specific set, the fractional accuracy of each pixel position included in the first set is interpolated and changed to the same value, and the fractional accuracy is changed. Calculating the pixel value of the first pixel position by adding the pixel values of the pixel positions and interpolating the added pixel values using a filter having a filter coefficient determined according to the changed fractional accuracy;
The image encoding method according to claim 1, wherein: A motion compensated prediction step for generating a motion compensated prediction image predicted based on a motion vector from a reference image;
An interpolated image generation step of generating a bi-directional prediction image obtained by interpolating the plurality of motion compensation prediction images generated for the plurality of reference images;
A decoding step of decoding the difference from encoded data obtained by encoding a difference between the input image and the bidirectional prediction image, and generating the input image based on the decoded difference and the bidirectional prediction image; Including
In the interpolation image generation step, a first set representing a combination of pixel positions of a plurality of motion compensated prediction images for obtaining a pixel value of a first pixel position of the bidirectional prediction image is a predetermined pixel. When included in a specific set which is a set of positions, the pixel value of each pixel position included in the first set is added, and the pixel value of the first pixel position is calculated by interpolating the added pixel value ,
An image decoding method characterized by the above. When the first set is not included in the specific set, the interpolation image generation step calculates an interpolation value obtained by interpolating the pixel value for each pixel value of each pixel position included in the first set, and the interpolation Calculating a pixel value of the first pixel position by adding the values;
The image decoding method according to claim 9. In the interpolation image generation step, when the first set is not included in the specific set, the tap length of at least some of the filters used for calculating the interpolation value is made smaller than other filters.
The image decoding method according to claim 10. In the interpolation image generation step, when the first set is not included in the specific set, the first set is replaced with one of the specific sets determined in advance according to the first set. Calculating the pixel value of the first pixel position by adding the pixel values of each pixel position included in the specific set and interpolating the added pixel values;
The image decoding method according to claim 10. The specific set is a set of pixel positions having the same fractional precision;
The image decoding method according to claim 9. The specific set is a set of pixel positions whose fractional accuracy is symmetric with respect to 1/2,
In the interpolation image generation step, when the first set is included in the specific set, one of the pixel positions included in the first set is inverted with respect to the first pixel position. Adding the pixel value and the pixel value of the other pixel position among the pixel positions included in the first set, and calculating the pixel value of the first pixel position by interpolating the added pixel value;
The image decoding method according to claim 9. The specific set is a set of pixel positions whose fractional accuracy is ½,
In the interpolation image generation step, when the first set is included in the specific set, the pixel value of each pixel position included in the first set and the same filter coefficient as each pixel position included in the first set Calculating the pixel value at the first pixel position by interpolating the pixel value at the pixel position using
The image decoding method according to claim 9. The specific set is a set of pixel positions with different fractional accuracy,
In the interpolation image generation step, when the first set is included in the specific set, the fractional accuracy of each pixel position included in the first set is interpolated and changed to the same value, and the fractional accuracy is changed. Calculating the pixel value of the first pixel position by adding the pixel values of the pixel positions and interpolating the added pixel values using a filter having a filter coefficient determined according to the changed fractional accuracy;
The image decoding method according to claim 9. A motion-compensated prediction unit that generates a motion-compensated prediction image that is predicted based on a motion vector from a reference image;
An interpolated image generating unit that generates a bi-directional prediction image obtained by interpolating the plurality of motion compensation prediction images generated for the plurality of reference images;
An encoding unit that generates encoded data obtained by encoding the difference between the input image and the bidirectional prediction image,
In the interpolation image generation unit, a first set representing a combination of pixel positions of a plurality of motion compensated prediction images for obtaining a pixel value of a first pixel position of the bidirectional prediction image is a predetermined pixel. When included in a specific set which is a set of positions, the pixel value of each pixel position included in the first set is added, and the pixel value of the first pixel position is calculated by interpolating the added pixel value ,
An image encoding device characterized by the above. A motion-compensated prediction unit that generates a motion-compensated prediction image that is predicted based on a motion vector from a reference image;
An interpolated image generating unit that generates a bi-directional prediction image obtained by interpolating the plurality of motion compensation prediction images generated for the plurality of reference images;
A decoding unit that decodes the difference from encoded data obtained by encoding a difference between the input image and the bidirectional prediction image, and generates the input image based on the decoded difference and the bidirectional prediction image; With
In the interpolation image generation unit, a first set representing a combination of pixel positions of a plurality of motion compensated prediction images for obtaining a pixel value of a first pixel position of the bidirectional prediction image is a predetermined pixel. When included in a specific set which is a set of positions, the pixel value of each pixel position included in the first set is added, and the pixel value of the first pixel position is calculated by interpolating the added pixel value ,
An image decoding apparatus characterized by the above.

Images