WO2011145865A2 - Apparatus and method for constructing and indexing a reference image - Google Patents

Abstract

The present invention relates to an apparatus and method for constructing and indexing a reference image. The apparatus for constructing and indexing a reference image according to the present invention constructs and indexes a reference image so as to estimate a motion vector, wherein the apparatus comprises: a reference-image-constructing unit which interpolates the reference image to a variety of spatial resolutions and arranges a reference frame in accordance with the temporal position of the reference frame; and an index allocation unit which allocates a reference frame index in accordance with the temporal position of the reference frame.

Description

Reference image construction and indexing device and method

An embodiment of the present invention relates to a reference image construction and indexing apparatus and method. More specifically, the reference image may be interpolated with various spatial resolutions, or may be a reference image on which brightness compensation, zoom-in and zoom-out, warping, etc. are performed. The present invention relates to an apparatus and method for constructing and indexing a reference picture for assigning a reference frame index according to a temporal position.

In an existing video codec such as JM or KTA, when performing inter coding, a motion vector is estimated using a plurality of previous frames or a plurality of future frames as a reference picture. This use of multiple reference images increases the likelihood of finding a block similar to the current block, and enables accurate motion estimation in situations where scene changes or obscured objects protrude.

1 shows the advantage of using a plurality of reference images. If the previous reference is used for the previous one, the '19' mark is present on the player's uniform with the ball, but not in the previous reference. In this case, it is difficult to find MV, which reduces coding efficiency. On the other hand, if the reference image is found in the past reference image rather than the previous chapter, the mark '19' can be found. Therefore, we can find the exact motion vector when coding the current chapter.

In this case, when a motion vector is estimated using a plurality of reference images, a reference frame index indicating a referenced image should be encoded every code of each block. Currently, JM software uses a unary coding scheme when coding index information of a reference picture. The binary coding scheme is shown in FIG. 2 and Table 1. FIG.

TABLE 1

In general, the image most similar to the current frame is the image closest to the current frame in time. Therefore, in general, the reference frame index assigns a short codeword to a reference frame close to the current frame based on a temporal order, and assigns a long codeword to a reference picture that is far apart in time. Such a method can be used as shown in FIG. 3 and Table 2 for bidirectional prediction.

TABLE 2

Meanwhile, a process of encoding the motion vector estimated by the video codec is as follows. First, a prediction motion vector (PMV) predicted from neighboring blocks of the current block is calculated, and then a difference vector between the PMV and the motion vector found for the current block is calculated. The difference vector is encoded into variable length codewords arranged by integer multiples of the motion vector resolution considered in the codec.

In JM software, which is a conventional video codec, the motion vector is estimated in integer pixel units, half pixel (1/2) units, and quarter pixel units, and the signal is compressed using the motion vector having the most efficient resolution. It was. Meanwhile, in the KTA software, in order to estimate a more accurate motion vector, a more precise motion can be found by estimating a motion vector from an integer pixel unit to 1/8 pixel unit. However, since the KTA codec needs to encode motion vectors of all resolutions such as integer pixel units, 1/2 pixel units, 1/4 pixel units, and 1/8 pixel units, even when encoding small size motion vectors You must use codewords. As a result, the amount of generated bits increases, resulting in a decrease in compression efficiency. This problem is described in detail as follows.

Table 3 exemplarily shows codewords allocated to differential motion vectors in a codec considering integer pixel units to quarter pixel units.

TABLE 3

As can be seen from Table 3, in the conventional compression standard, when all motion vectors of various resolutions are encoded, long codewords are used to encode small motion vectors, and as a result, they are generated in the process of encoding motion vectors. The size of the data to be increased increases the compression efficiency. For example, in Table 3, when the differential motion vector is (3,2), the bitstring '000011000' having the code number '23' is used to encode '3', and the '2' is encoded. Bit string '000010000' with code number '15' is used. As long codewords are used to encode a motion vector of a small size, encoding is performed considering both a codeword for encoding a motion vector of 1/2 pixel unit and a 1/4 pixel unit and a motion vector of integer pixel unit. This is because the codewords used together are used together.

As described above, estimating a high resolution motion vector has the advantage of finding a reference block that is highly correlated with the current coding block, but all resolution vectors from low resolution motion vector values to high resolution vector values Since the variable length codeword is considered, the compression efficiency may be lowered. For example, if most blocks in a particular frame can be encoded using only integer vectors or motion vectors of 1/2 pixel unit, variable length considering all resolutions from integer pixel unit to 1/8 pixel unit If a codebook is used, the length of the variable length codewords of frequently used integer pixels and half-pixel codevectors is increased due to the codewords considering unused 1/4 pixels and 1/8 pixels. The efficiency can be lowered. In such a case, a motion vector of all resolutions is not required in a frame or a coding block, and only a few kinds of resolution vectors can be used for efficient coding.

On the contrary, in other cases, the compression efficiency may be high when variable length codewords considering motion vectors of all resolutions from integer pixel units to 1/8 pixel units are used because of characteristics of internal pixel values in a specific frame and block.

As another example, in a specific frame and block, using a variable length codebook considering only some specific resolutions (for example, 1/2 pixel unit and 1/8 pixel unit) due to characteristics of internal pixel values increases compression efficiency. Can be. In this case, it is possible to increase the compression efficiency by not supporting the codewords for the integer pixel unit and the quarter pixel unit motion vector in the variable length codebook.

In order to solve the problem of deterioration of the compression rate related to the resolution of the motion vector, an embodiment of the present invention may interpolate reference images with different spatial resolutions and use them as reference images having different reference frame indices. It is an object to provide a configuration and indexing apparatus and method. The reference picture structure and the reference picture indexing apparatus and method according to an embodiment of the present invention do not limit the interpolation method or the motion estimation method.

In addition, the reference images may be configured based on the differentiated spatial resolution for each image, but in addition, the brightness-compensated reference image, the zoom-in and zoom-out reference image, and the warped reference An object of the present invention is to provide a reference picture structure and indexing device that can be used as a reference picture by variously configuring the picture.

In the reference picture structure and the reference picture indexing device according to the first embodiment of the present invention, in the reference picture structure and the indexing device for constructing and indexing a reference picture to estimate a motion vector, the reference picture is interpolated with different spatial resolutions. a reference picture construction unit arranged to arrange reference frames by interpolation; And an index allocator for allocating a reference frame index according to the spatial resolution of the reference frame.

Here, the reference frame is a frame which is not interpolated, 1/2 pixel frame, 1/4 pixel frame, 1/8 pixel frame, 1/2 pixel frame, 1/4 pixel frame and 1/8 pixel frame. Two or more of them may be any one frame of the combined frame.

In addition, when the motion vector estimated in the reference frame is encoded, the motion vector may be encoded using a different codebook according to the spatial resolution of the reference frame.

In the reference image construction and indexing apparatus according to the second embodiment of the present invention, in the reference image construction and indexing apparatus for constructing and indexing a reference image for estimating a motion vector, the reference image is zoomed at various magnifications. A reference image construction unit for arranging reference frames according to zooming magnifications; And an index allocator for allocating a reference frame index according to the zooming magnification of the reference frame.

Here, the reference frame may be any one of a frame not zoomed, a frame zoomed in, and a frame zoomed out.

In addition, when estimating a motion vector in a reference frame, the motion vector may be estimated using different variable length codebooks according to the zooming magnification of the reference frame.

In the reference image construction and indexing apparatus according to the third embodiment of the present invention, in the reference image construction and indexing apparatus for constructing and indexing a reference image for estimating a motion vector, a type of warping by warping the reference image A reference image configuring unit for arranging reference frames according to one another; And an index allocator for allocating a reference frame index according to the type of warping of the reference frame.

Here, the reference frame may be any one of a frame that is not warped and a frame that is warped.

In addition, when estimating the motion vector in the reference frame, if the reference frame is a warped frame, the motion may be encoded by estimating the warping parameter and the motion vector.

In the reference image construction and indexing apparatus according to the fourth embodiment of the present invention, in the reference image construction and indexing apparatus for constructing and indexing a reference image for estimating a motion vector, the reference image is subjected to brightness compensation to compensate for the degree of brightness compensation. A reference image configuring unit for arranging reference frames according to a method; And an index allocator for allocating the reference frame indexes according to the brightness compensation degree and the method of the reference frame.

Here, the reference frame may be any one of a frame not subjected to brightness compensation and a frame subjected to brightness compensation.

In addition, when estimating a motion vector from the reference frame, if the reference frame is a frame that has been subjected to brightness compensation, the motion vector may be estimated with respect to the brightness compensated reference image data.

In the reference image construction and indexing apparatus according to the fifth embodiment of the present invention, in the reference image construction and indexing apparatus for constructing and indexing a reference image for estimating a motion vector, interpolation processing for interpolating the reference image with various spatial resolutions A reference image construction unit for arranging reference frames according to a processing method using at least two of zooming processing, warping processing, and brightness compensation processing for zooming at various magnifications; And an index allocator for allocating a reference frame index according to the method of processing the reference frame.

On the other hand, the reference image construction and indexing apparatus according to the first embodiment, in the reference image configuration and indexing method for constructing and indexing the reference image to estimate the motion vector, interpolation resolution by interpolating the reference image with different spatial resolutions Arranging a reference frame according to; And allocating a reference frame index according to the interpolation resolution of the reference frame.

In addition, the reference image construction and indexing apparatus according to the second embodiment is a reference image construction and indexing method for constructing and indexing a reference image in order to estimate a motion vector. Arranging reference frames accordingly; And assigning a reference frame index according to the zooming magnification of the reference frame.

 Also, in the reference picture construction and indexing method of constructing and indexing a reference picture for estimating a motion vector, the reference picture construction and indexing apparatus according to the third embodiment includes a reference frame according to a warping method by warping the reference picture. Arranging; And assigning a reference frame index according to the warping method of the reference frame.

 In addition, the reference image construction and indexing apparatus according to the fourth embodiment is a reference image construction and indexing method for constructing and indexing a reference image for estimating a motion vector. Arranging a reference frame according to; And allocating a reference frame index according to a method and a degree of brightness compensation of the reference frame.

 In addition, the reference image construction and indexing apparatus according to the fifth embodiment includes a reference image construction and indexing method for constructing and indexing a reference image to estimate a motion vector, the interpolation process of interpolating a reference image at various spatial resolutions, Arranging reference frames according to a processing method using at least two of zooming processing, warping processing, and brightness compensation processing for zooming at various magnifications; And assigning a reference frame index according to the processing method of the reference frame.

In addition, in the reference image indexing technique according to the present invention, the forward and backward reference pictures may be referred to as consecutive index numbers without separately displaying the forward and backward reference pictures as separate flags. have.

The reference image configuration method described in the above-described embodiments and the invention may be notified to the decoder using separate information, and in this case, it may be notified through a sequence header, a picture header, or a slice header. In addition, when the encoder and the decoder configure the reference image by the promised method, the decoder may configure and decode the reference image without transmitting additional information.

According to an embodiment of the present invention, a reference in which a reference image is interpolated with various spatial resolutions, brightness compensation processing, zoom-in and zoom-out processing, and warping processing are performed. It is possible to improve the coding efficiency of the motion vector by configuring a video and assigning a reference frame index according to a processing method.

1 is a diagram illustrating an example of MV search when a plurality of reference images are used.

 2 is a diagram illustrating an example of assigning a reference frame index in the case of using the past frames as reference pictures.

 3 is a diagram illustrating an example of reference frame index allocation when using past and future frames as reference pictures.

 4 is a diagram schematically illustrating a reference image construction and indexing apparatus according to an embodiment of the present invention.

 5 is a flowchart illustrating a method of constructing and indexing a reference image by the apparatus for constructing and indexing a reference image of FIG. 4.

 6 is a diagram illustrating an example of reference frame index allocation according to a spatial resolution of a reference frame.

 7 is a diagram illustrating another example of reference frame index allocation according to a spatial resolution of a reference frame.

 8 illustrates an example of reference frame index allocation according to zoom in / out of a reference frame.

 9 is a diagram illustrating another example of reference frame index allocation according to zoom in / out of a reference frame.

 10 illustrates an example of reference frame index allocation according to a warping method of a reference frame.

 11 is a diagram illustrating another example of reference frame index allocation according to a warping method of a reference frame.

 12 is a diagram illustrating an example of a reference frame index allocation method according to a brightness compensation method of a reference frame.

 13 is a diagram illustrating another example of a method of allocating a reference frame index according to the brightness compensation method of a reference frame.

 14 is a diagram illustrating an example of indexing forward and backward reference images with consecutive numbers without using a separate flag for information on the direction when predicting a reference picture frame in both directions.

FIG. 15 is a diagram illustrating an example of indexing forward and backward reference images using consecutive numbers without using a separate flag for information on the direction when predicting a reference picture frame in both directions.

4 is a diagram schematically illustrating a reference image construction and indexing apparatus according to an embodiment of the present invention. Referring to FIG. 4, the reference image constructing and indexing apparatus 400 may include a reference image constructing unit 410 and an index allocating unit 420. The reference picture construction and indexing apparatus 400 may be included as an independent component of the inter predictor in the image encoding apparatus, or may be implemented as an additional function of the inter predictor in the image encoding apparatus. The reference image construction and indexing apparatus 400 constructs and indexes a reference image to estimate a motion vector.

The reference image configuring unit 410 interpolates the reference image with various spatial resolutions and arranges the reference frames according to the spatial resolutions. Alternatively, the reference image configuration unit 410 may zoom out the reference image at various magnifications and arrange the reference frames according to the zooming magnification. Alternatively, the reference image configuration unit 410 may warp the reference image to arrange the reference frames according to the warping method. Alternatively, the reference image configuring unit 410 may arrange the reference frames according to the brightness compensation method and degree by performing brightness compensation on the reference image.

The index allocator 420 allocates a reference frame index according to a processing method and a degree of the reference frames arranged by the reference image configuring unit 410.

 5 is a flowchart illustrating a method of constructing and indexing a reference image by the apparatus for constructing and indexing a reference image of FIG. 4. With reference to the drawings will be described in detail the function and operation of the reference image configuration and indexing apparatus according to an embodiment of the present invention.

The reference image configuring unit 410 performs at least one of interpolation processing of various spatial resolutions, zooming processing at various magnifications, warping processing, and brightness compensation processing with respect to the reference image, thereby reconstructing the reference frame according to a processing method and a degree of processing. Arrange (S510).

The index allocator 420 allocates the reference frame index to the reference frames arranged according to the processing method and the degree of processing by the reference image configuring unit 410 (S520).

6 is a diagram illustrating an example of reference frame index allocation according to a spatial resolution of a reference frame.

In FIG. 6, 'n-1 frame' means a past n-1th frame that has not been processed. In addition, '1/2 pixel frame', '1/4 pixel frame', and '1/8 pixel frame' interpolate an existing reference image, and each of them is 1/2 pixel, 1/4 A reference image in which only information in pixel units and 1/8 pixel units is selected and stored is shown. For example, the "1/8 pixel frame of the n-1 image" is a reference image that stores only a signal value of 1/8 pixel unit after interpolating the n-1 th reference image. In addition, "1/2 & 1/4 pixel frame of image n-2" refers to a reference image that stores only signal values of 1/2 pixel unit and 1/4 pixel unit after interpolating the n-2th reference image. to be. The reference frame index of FIG. 6 may be encoded using Table 1.

 In the case of configuring the reference image in the same manner as in FIG. 6, a method of encoding a motion vector for a coding block of a current frame is as follows. When estimating the motion vector of the current coding block in reference index 1,2,3 frames, the motion vector is estimated using a codebook considering all integer pixels, 1/2 pixels, and 1/4 pixel units. When estimating the motion vector in the a1 frame, the motion vector is estimated only in units of 1/8 pixels. When estimating the motion vector only in the a2 frame, the motion vector is estimated only in 1/2 pixel units and 1/4 pixel units. The codebook for encoding the motion vector in each frame uses different ones designed for each. In reference index 1,2,3 frames, 'integer', '1/2', '1/4', ' Codebook considering motion vectors of all resolutions such as 1/8 'is used.

TABLE 4

When encoding a motion vector estimated in the reference frame a1, a codebook for encoding only a motion vector in units of 1/8 pixels is used as shown in Table 5.

TABLE 5

In addition, when encoding the motion vector found in the reference frame a2, a codebook for encoding code vectors of only 1/2 pixel units and 1/4 pixel units is used as shown in Table 6.

TABLE 6

Tables 4 to 6 are merely examples of a codebook for representing a motion vector according to the resolution information, and embodiments of the present invention do not limit the method of representing the motion vector. 6 and the aforementioned Tables 4 to 6 are merely examples of the present invention, and do not limit the type of spatial resolution signal for each reference frame position (Reference index number). That is, another embodiment of the present invention may be implemented as shown in FIG. 7, and this embodiment is also not limited to merely an example of the present invention. This embodiment of the present invention can also be used in bidirectional motion prediction.

8 illustrates an example of reference frame index allocation according to zoom in / out of a reference frame. In FIG. 8, n-1, n-2, and n-3 frames refer to existing n-1 th, n-2 th, n-3 th reference frames without any processing. An 'A1 times zooming frame of the n-1 frame' is an image made by zooming the n-1th reference picture A1 times. At this time, if the value of A1 is larger than 1 (that is, A1> 1), the image is zoomed in. If the value of A1 is smaller than 1 (ie, A1 <1), the image is zoomed out. . The embodiment of the present invention does not limit the method of zooming in or zooming out. The reference frame index of FIG. 8 is encoded using Table 1.

 In the case of configuring the reference image in the manner illustrated in FIG. 8, a method of encoding a motion vector for a coding block of the current frame is as follows. The motion vector of the current coding block is estimated by estimating the motion vector considering all integer pixels, 1/2 pixels, 1/4 pixels, and 1/8 pixel units (up to a predetermined resolution) in the reference index 1,2,3 frames, In frame a1, a motion vector is estimated for A1-times zoomed image data. In the frame a2, the motion vector is estimated on the A2-times zoomed image data.

 8 illustrates an embodiment of the present invention, and the type of the zoom in / zoom out signal for each reference frame position is not limited. That is, according to an exemplary embodiment of the present invention, the reference image is included in the reference image list as the reference image. Another example of such an invention may be represented as shown in FIG. 9, and FIG. 9 is also merely an example of the present invention and does not limit the present invention. In addition, embodiments according to the present invention may be used in bidirectional motion prediction.

 10 illustrates an example of reference frame index allocation according to a warping method of a reference frame. In FIG. 10, n-1, n-2, and n-3 frames refer to existing n-1 th, n-2 th, n-3 th reference frames without any processing. The B1 warping frame of the n-1 frame is an image in which the n-1th reference image is warped by the B1 method. The warping method includes a method of modifying a pixel value and a block shape by performing a linear transformation and an equilibrium shift process on pixel values in a block, but embodiments of the present invention do not limit the warping method. The reference frame index of FIG. 10 may be encoded using Table 1.

In the case of configuring the reference image in the same manner as in the embodiment of FIG. 10, a method of encoding a motion vector for a coding block of a current frame is as follows. The motion vector of the current coding block is estimated by estimating the motion vector considering all integer pixels, 1/2 pixels, 1/4 pixels, and 1/8 pixel units (up to a predetermined resolution) in the reference index 1,2,3 frames, In frame a1, a motion vector is estimated for image data processed using the B1 warping method. In frame a2, the motion vector is estimated with respect to the processed image data using the B2 warping method.

10 illustrates an embodiment of the present invention, and the type of warping method for each reference frame position is not limited. That is, the method proposed in the embodiment according to the present invention includes an image warped in various ways by including one reference image as a reference image in the reference image list. Another example of such an invention may be represented as shown in FIG. 11, and FIG. 11 is also merely an example of the present invention and does not limit the present invention. In addition, embodiments according to the present invention may be used in bidirectional motion prediction.

12 is a diagram illustrating an example of a reference frame index allocation method according to a temporal position and brightness compensation method of a reference frame. In FIG. 12, n-1, n-2, and n-3 frames mean existing n-1 th, n-2 th, n-3 th reference frames without any processing. The W1 brightness compensation frame of the n-1 frame is an image obtained by performing a brightness compensation process on the n-1th reference picture using the W1 method. Brightness compensation methods include a method of gaining by varying the magnification of illuminance, a method of offsetting brightness, and the like, but the embodiment of the present invention does not limit the method of compensating brightness. The reference frame index of FIG. 12 may be encoded using Table 1.

In the case of configuring a reference image in the same manner as in FIG. 12, a method of encoding a motion vector for a coding block of a current frame is as follows. The motion vector of the current coding block is estimated by estimating the motion vector considering all integer pixels, 1/2 pixels, 1/4 pixels, and 1/8 pixel units (up to a predetermined resolution) in the reference index 1,2,3 frames, In frame a1, the motion vector is estimated for the processed image data using the W1 brightness compensation method. In frame a2, the motion vector is estimated for the processed image data using the W2 brightness compensation method.

12 illustrates an embodiment of the present invention, and the type of the brightness compensation method for each reference frame position is not limited. That is, the method proposed in the embodiment of the present invention includes the reference image including the image compensated for brightness by various methods in the reference image list as the reference image. Another example of such an invention may be represented as shown in FIG. 13, and FIG. 13 is also merely an example of the present invention and does not limit the present invention. In addition, embodiments according to the present invention may be used in bidirectional motion prediction.

14 is a diagram illustrating an example of displaying past and future reference images as consecutive index numbers without a separate flag bit indicating direction information when a reference frame is predicted in both directions. In FIG. 14, reference frames of n-1 and n + 1 that are close in time are continuously assigned low index numbers, and relatively high index numbers are assigned to n-2 and n + 2 that are far in time.

14 illustrates an embodiment of the present invention, and the type of a method of allocating a reference frame index in bidirectional prediction is not limited. That is, the limiting method in the embodiment of the present invention includes an example of allocating a low index to a reference image that is close in time. For example, as shown in FIG. 15, FIG. 15 is also merely an example of the present invention and does not limit the present invention.

The above embodiments have described a case in which a specific reference image is used as an additional reference image by 'resolution conversion', 'zooming', 'warping', and 'brightness conversion'. However, the present invention is not limited to the reference image transformation using the four methods, but may use various additional transformation methods for accurately finding the motion vector.

In addition, the decoder of the present invention may make an appointment with the encoder in advance of a method of configuring a specific reference image to configure the reference image by itself without additional information, and the encoder delivers additional information about the reference image construction method to the decoder for reference. You can see how the image is constructed. The coding method of information transmitted to the decoder is not limited.

In addition, the techniques proposed in the embodiment of the present invention do not restrict the use of images modified by one or more methods in the list of reference images as a reference image. For example, among reference images, 'unmodified past reference image', 'brightness compensated reference image', 'warped reference image', 'zoomed reference image', 'reference image composed of specific resolution information', The reference image modified by the additional processing method may be used together. In this case, the method of allocating the reference index of the processed images is not limited in the embodiment of the present invention.

In addition, the use of Table 1 to encode the reference frame index in the description of the present invention is only one example, and the present invention does not limit the method of encoding the reference frame index.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention. In addition, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, the protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope will be construed as being included in the scope of the present invention.

As described above, in the embodiment of the present invention, the reference image may be variously configured by performing interpolation processing at various spatial resolutions or zooming, zooming, and brightness compensation processing at various magnifications. It is a very useful invention to generate an effect of improving the coding efficiency of a motion vector by allocating a reference frame index according to a processing method and a degree.

CROSS-REFERENCE TO RELATED APPLICATION

 This patent application is related to the patent application No. 10-2010-0046178 filed in Korea on May 17, 2010 and the patent application No. 10-2011-0045528 filed in Korea on May 16, 2011. If priority is claimed under section (a) (35 USC §119 (a)), all of this is incorporated by reference into this patent application. In addition, if this patent application claims priority for the same reason as for the above-mentioned countries, all the contents are incorporated in this patent application by reference.

Claims (38)

In the reference picture construction and indexing apparatus for constructing and indexing a reference picture to estimate a motion vector,  A reference image constructing unit for processing the reference image in various arbitrary methods and arranging reference frames according to the processing method; And  An index allocator for allocating a reference frame index according to a method processed by the reference image constructing unit  Reference image construction and indexing apparatus comprising a. The method of claim 1, When encoding the motion vector estimated in the reference frame, the reference image configuration and indexing apparatus, characterized in that for encoding the motion vector using different codebooks according to the method processed by the reference image configuration unit.  In the reference picture construction and indexing apparatus for constructing and indexing a reference picture to estimate a motion vector,  A reference image construction unit for interpolating the reference image with various spatial resolutions and arranging reference frames according to interpolation resolutions; And  An index allocator for allocating a reference frame index according to the interpolation resolution of the reference frame  Reference image construction and indexing apparatus comprising a.  The method of claim 3, wherein The reference frame is one of an uninterpolated frame, a 1/2 pixel frame, a 1/4 pixel frame, a 1/8 pixel frame, and a 1/2 pixel, 1/4 pixel, and 1/8 pixel unit. The reference picture construction and indexing device, characterized in that any two or more frames are any one of the combined frame.  The method of claim 3, wherein And a method of encoding a motion vector estimated in the reference frame, the motion vector is encoded using a different codebook according to the spatial resolution of the reference frame.  In the reference picture construction and indexing apparatus for constructing and indexing a reference picture to estimate a motion vector,  A reference image configuration unit for zooming the reference image at various magnifications and arranging reference frames according to zooming magnifications; And  An index allocator for allocating a reference frame index according to the zooming magnification of the reference frame  Reference image construction and indexing apparatus comprising a.  The method of claim 6,  And the reference frame is any one of a frame not zoomed, a frame zoomed in, or a frame zoomed out.  The method of claim 6, When encoding the motion vector estimated in the reference frame, the reference image configuration and indexing apparatus, characterized in that for encoding the motion vector using a different codebook according to the zooming magnification of the reference frame.  In the reference picture construction and indexing apparatus for constructing and indexing a reference picture to estimate a motion vector,  A reference image configuration unit for arranging reference frames according to a warping method by warping the reference image; And  An index allocator for allocating a reference frame index according to the warping method of the reference frame  Reference image construction and indexing apparatus comprising a.  The method of claim 9,  And the reference frame is any one of a frame that is not warped and a frame that is warped.  The method of claim 9,  When encoding the motion vector estimated in the reference frame, the reference image configuration and indexing apparatus, characterized in that for encoding the motion vector using different codebooks according to the warping method of the reference frame.  In the reference picture construction and indexing apparatus for constructing and indexing a reference picture to estimate a motion vector,  A reference image configuration unit for arranging reference frames according to a brightness compensation method and degree by performing brightness compensation on the reference image; And  An index allocator for allocating a reference frame index according to the brightness compensation method and degree of the reference frame  Reference image construction and indexing apparatus comprising a.  The method of claim 12,  And the reference frame is any one of a frame not subjected to brightness compensation and a frame subjected to brightness compensation.  The method of claim 12, When encoding the motion vector estimated in the reference frame, the reference image configuration and indexing apparatus, characterized in that for encoding the motion vector using a different codebook according to the brightness compensation and degree of the reference frame.  In the reference picture construction and indexing apparatus for constructing and indexing a reference picture to estimate a motion vector,  A reference for arranging reference frames according to a processing method and a degree by performing a combination process combining at least two of interpolation processing for interpolating the reference image at various spatial resolutions, zooming processing at various magnifications, warping processing, and brightness compensation processing. An image component; And  An index allocator for allocating a reference frame index according to the processing method and degree of the reference frame  Reference image construction and indexing apparatus comprising a.  In the reference picture construction and indexing apparatus for constructing and indexing a reference picture to estimate a motion vector,  A reference picture configuration unit for arranging reference frames according to a temporal position and a degree when bidirectionally predicting the reference picture; And  Index allocator for freely allocating the reference frame index according to the temporal position and degree when bidirectionally predicting the reference image  Reference image construction and indexing device comprising a.  The method of claim 16, wherein the index allocation unit,  And a frame is indexed while separately encoding the direction information of the reference frame.  The method of claim 16,  And a method for constructing the reference frame comprises encoding information about the reference frame.  The method of claim 16, wherein the index allocation unit, A reference image construction and indexing device, characterized in that to assign a low index to a reference image that is close in time.  A reference picture construction and indexing method for constructing and indexing a reference picture to estimate a motion vector,  Processing the reference image by various arbitrary methods and arranging reference frames according to the processing method; And  Allocating a reference frame index according to the processed method  Reference image construction and indexing method comprising a. The method of claim 20, And a method of encoding a motion vector estimated in the reference frame, encoding the motion vector using different codebooks according to the processed method.  A reference picture construction and indexing method for constructing and indexing a reference picture to estimate a motion vector,  Arranging reference frames according to spatial resolution by interpolating the reference image with various spatial resolutions; And  Allocating a reference frame index according to the spatial resolution of the reference frame  Reference image construction and indexing method comprising a.  The method of claim 22, The reference frame is one of an uninterpolated frame, a 1/2 pixel frame, a 1/4 pixel frame, a 1/8 pixel frame, and a 1/2 pixel, 1/4 pixel, and 1/8 pixel unit. A reference picture construction and indexing method, characterized in that any one or more frames of two or more combined frames.  The method of claim 22, And when the motion vector estimated in the reference frame is encoded, the motion vector is encoded using a different codebook according to the spatial resolution of the reference frame.  A reference picture construction and indexing method for constructing and indexing a reference picture to estimate a motion vector,  Zooming the reference image at various magnifications and arranging reference frames according to zooming magnifications; And  Allocating a reference frame index according to the zooming magnification of the reference frame  Reference image construction and indexing method comprising a.  The method of claim 25,  And the reference frame is any one of a frame not zoomed, a frame zoomed in, or a frame zoomed out.  The method of claim 25, And when the motion vector estimated in the reference frame is encoded, the motion vector is encoded using a different codebook according to the zooming magnification of the reference frame.  A reference picture construction and indexing method for constructing and indexing a reference picture to estimate a motion vector,  Arranging reference frames according to a warping method by warping the reference image; And  Allocating a reference frame index according to the warping method of the reference frame  Reference image construction and indexing method comprising a.  The method of claim 28,  And the reference frame is any one of a frame that is not warped and a frame that is warped.  The method of claim 28, And when the motion vector estimated in the reference frame is encoded, the motion vector is encoded using a different codebook according to the warping method of the reference frame.  A reference picture construction and indexing method for constructing and indexing a reference picture to estimate a motion vector,  Arranging reference frames according to a brightness compensation method and degree by performing brightness compensation on the reference image; And  Allocating a reference frame index according to the brightness compensation method and degree of the reference frame  Reference image construction and indexing method comprising a.  The method of claim 31, wherein  And the reference frame is any one of a frame not subjected to brightness compensation and a frame subjected to brightness compensation.  The method of claim 31, wherein When encoding the motion vector estimated in the reference frame, the reference image configuration and indexing method, characterized in that for encoding the motion vector using a different codebook according to the brightness compensation method and degree of the reference frame.  A reference picture construction and indexing method for constructing and indexing a reference picture to estimate a motion vector,  Arranging reference frames according to a processing method and a degree by performing a combination process combining at least two of interpolation processing for interpolating the reference image at various spatial resolutions, zooming processing at various magnifications, warping processing, and brightness compensation processing; ; And  Allocating a reference frame index according to a processing method and a degree of the reference frame  Reference image construction and indexing method comprising a.  A reference picture construction and indexing method for constructing and indexing a reference picture to estimate a motion vector,  Arranging reference frames according to a temporal position and a degree in bidirectional prediction of the reference image; And  Freely allocating a reference frame index according to a temporal position and a degree when bidirectionally predicting the reference image  Reference image construction and indexing method comprising a.  The method of claim 35, wherein  And a frame is indexed while separately encoding the direction information of the reference frame.  The method of claim 35, wherein  And a method for constructing the reference frame comprises encoding information about the reference frame.  The method of claim 35, wherein  The index allocation step, A reference image construction and indexing method comprising assigning a low index to a reference image that is close in time.

Images