JP4383367B2 - Motion vector detection device - Google Patents

Description

  The present invention relates to a motion vector detection technique, and, for example, to an apparatus for detecting a motion vector between frames in order to encode a moving image by an image encoding method including an inter-frame prediction mode.

In MPEG (Motion Picture Experts Group) -4, which is a standard for video compression coding, motion compensation prediction coding using motion vectors is performed. As a technique for detecting a motion vector, a reduced image is generated by reducing the resolution of the encoding target image, an approximate motion vector is detected using the reduced image, and then the original motion while referring to the approximate motion vector. A hierarchical motion vector detection method has been proposed in which a motion vector is detected using an image with a resolution of (for example, see Patent Document 1).
Japanese Patent Laid-Open No. 11-260215

  Conventionally, in the hierarchical motion vector detection method described above, every time a macroblock to be encoded changes, the image of the search area of the reference image is read from the frame memory to generate a reduced image. Congestion became a bottleneck, and the processing speed was limited.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique for shortening the time required for motion vector detection.

One embodiment of the present invention relates to a motion vector detection device. The motion vector detection device is a motion vector detection device that detects a motion vector between a first image and a second image that is a reference image of the first image, and acquires the second image; A reduced image generation unit that generates a reduced image having a resolution lower than that of the original image, a reduced image holding unit that stores at least a part of the reduced image, and a calculation unit that calculates a matching between the images and detects a motion vector The calculation unit refers to a motion vector obtained by calculating matching between the reduced image of the first image and the reduced image of the second image in units of macroblocks of a predetermined size. When the motion vector is detected by calculating the matching between the first image and the second image having the original resolution, and the motion vector is detected using the reduced image, the reduced image is selected from the reduced images. Portions held in the holding section is read out from the reduced image holding unit, wherein the reduced image holding unit is not held in the portion, characterized in that it is generated by the reduced image generation unit. According to such a configuration, it is not necessary to newly generate a reduced image held in the reduced image holding unit, so that the processing speed can be improved.

  The reduced image holding unit may include a memory capable of holding the entire reduced image of the second image, and store pixel data in the memory in an arrangement similar to the arrangement of pixels of the reduced image. By providing a memory with a sufficient capacity, the processing speed can be improved. Further, by making the arrangement of the pixel data on the memory the same as the arrangement of the pixels in the actual image, it becomes easy to control the reading and writing, and the reading and writing can be performed at high speed.

  The reduced image holding unit may store a region overlapping with a reduced image necessary for the next motion vector detection. Thereby, even when the capacity of the reduced image holding unit is small, the processing speed can be improved.

  Another aspect of the present invention relates to an image encoding device. The image encoding device is an image encoding device that encodes a moving image to generate an encoded data sequence, and between a first image and a second image that is a reference image of the first image, A motion vector detection unit that detects a motion vector; and an encoding unit that encodes the first image using the motion vector, wherein the motion vector detection unit obtains the second image, A reduced image generation unit that generates a reduced image having a resolution lower than that of the image, a reduced image holding unit that stores at least a part of the reduced image, a calculation unit that calculates a matching between the images and detects a motion vector, The calculation unit refers to a motion vector obtained by calculating a matching between the reduced image of the first image and the reduced image of the second image, and the first image having the original resolution Matchon of the second image When the motion vector is detected by calculating and the motion vector is detected using the reduced image, the portion of the reduced image held in the reduced image holding unit is read from the reduced image holding unit. It is characterized by that.

  Yet another embodiment of the present invention relates to a motion vector detection method. The motion vector detection method is a method of detecting a motion vector between a first image and a second image that is a reference image of the first image, and acquires the second image, Generating a reduced image having a resolution lower than that of the image; calculating a matching between the reduced image of the first image and the reduced image of the second image; and detecting an approximate motion vector; and A motion vector is detected by calculating matching between the first image and the second image of the original resolution with reference to a simple motion vector, and at least a part of the reduced image is held as a reduced image A step of storing in the reduced image holding unit, and when detecting a motion vector using the reduced image, a portion of the reduced image held in the reduced image holding unit holds the reduced image Characterized in that it is read from.

  It should be noted that any combination of the above-described constituent elements and a representation obtained by converting the expression of the present invention between a method, an apparatus, a system, a program and the like are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which shortens the time required for the detection of a motion vector can be provided.

  The image encoding apparatus according to the embodiment first generates a reduced image with a reduced resolution from an image to be encoded, and uses the reduced image to generate a rough motion vector with a low resolution (hereinafter referred to as an “approximate motion vector”). After the detection, the motion vector is detected using the original high-resolution image with reference to the approximate motion vector. The present embodiment proposes a technique for reducing the memory access amount and improving the motion vector detection speed when using such a hierarchical motion vector detection technique.

  FIG. 1 shows an overall configuration of an image encoding device 10 according to an embodiment of the present invention. The image encoding device 10 includes a motion vector detection circuit 24, a motion compensation circuit 26, a frame memory 28, an encoding circuit 30, a decoding circuit 32, an output buffer 34, a code amount control circuit 36, and a reference mode selection circuit 38. . These configurations can be realized in terms of hardware by a CPU, memory, or other LSI of any computer, and in terms of software, they are realized by programs loaded into the memory. It depicts the functional blocks that are realized. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  An image input from the outside to the image encoding device 10 (hereinafter referred to as “input image”) is sent to the motion vector detection circuit 24. The motion vector detection circuit 24 detects a motion vector between an input image and an image (hereinafter referred to as “reference image”) stored in advance in the frame memory 28 and used as a reference target for prediction. The motion compensation circuit 26 acquires the value of the quantization step used for quantization from the code amount control circuit 36, and determines the quantization coefficient and the macroblock prediction mode. The motion vector detected by the motion vector detection circuit 24, the quantization coefficient determined by the motion compensation circuit 26, and the macroblock prediction mode are sent to the encoding circuit 30. Also, the motion compensation circuit 26 sends the difference between the predicted value and the actual value for the macroblock to the encoding circuit 30 as a prediction error.

  The encoding circuit 30 encodes the prediction error using the quantization coefficient and sends it to the output buffer 34. The encoding circuit 30 sends the quantized prediction error and the quantized coefficient to the decoding circuit 32. The decoding circuit 32 decodes the quantized prediction error based on the quantization coefficient, and sends the sum of the decoded prediction error and the prediction value from the motion compensation circuit 26 to the frame memory 28 as a decoded image. This decoded image is sent to the motion vector detection circuit 24 as a reference image when it is referred to in the subsequent image encoding process. The code amount control circuit 36 acquires the state of the accumulation amount of the output buffer 34, and generates a quantization step value used for the next quantization according to the state of the accumulation amount.

  The reference mode selection circuit 38 switches the frame prediction mode among intra-frame coding, inter-frame forward prediction coding, and inter-frame bidirectional prediction coding, and provides frame prediction mode information to other circuits. Is output.

  FIG. 2 shows the internal configuration of the motion vector detection circuit. The motion vector detection circuit 24 includes a reduced image generation unit 40, a reduced image holding unit 42, and a calculation unit 44. The reduced image generation unit 40 acquires the input image and the reference image stored in the frame memory 28, reduces the resolution of these images, and generates an image reduced to a predetermined magnification. The reduced image generation unit 40 may reduce the image by thinning out pixels, average pixel data included in a block of a predetermined size, and use the block as one pixel, or use any technique. A reduced image may be generated. The calculation unit 44 calculates a difference (prediction error) between a macroblock to be encoded of the input image (encoding target macroblock) and a macroblock in the search area of the reference image, and the difference is the smallest. A motion vector is detected by searching a macroblock of the reference image. The reduced image holding unit 42 is configured by a memory such as an SDRAM, and holds a part or all of the reduced image generated by the reduced image generating unit 40, as will be described later. The reduced image holding unit 42 may be realized by a part of the frame memory 28 or may be shared with the frame memory 28.

  In the present embodiment, hierarchical motion vectors are detected. That is, first, an approximate motion vector in a reduced image with low resolution is detected, and a motion vector in an image with high resolution is detected while referring to the approximate motion vector. First, the reduced image generation unit 40 generates an encoding target macroblock of the input image and a reduced image of the search area of the reference image. Subsequently, the calculation unit 44 acquires the reduced encoding target macroblock and the search region, performs block matching, and detects an approximate motion vector. Next, the calculation unit 44 acquires the encoding target macroblock and the original image of the search area, performs block matching around the detected approximate motion vector, and detects a motion vector. The motion vector is detected by the calculation unit 44 in a predetermined size, for example, in units of macroblocks of vertical 16 pixels × horizontal 16 pixels.

  FIG. 3 is a diagram for explaining a conventional motion vector detection procedure. FIG. 3A shows the search area when detecting the motion vector of the macro block at the upper left corner of the input image. In the reference image 60a, a region 64a including a macroblock 62a located at the same position as the encoding target macroblock of the input image is set as a search region. In this example, one macroblock is 16 pixels long × 16 pixels wide, and the search area 64a is 80 pixels long × 80 pixels wide.

  The reduced image generation unit 40 reads out the pixel data of the encoding target macroblock of the input image from the frame memory 28, reduces the image by a factor of 1/4, and generates a reduced image of vertical 4 pixels × horizontal 4 pixels. In addition, the pixel data of the search area 64a is read from the frame memory 28, and the image is reduced by a factor of 1/4 to obtain a reduced image of 20 pixels vertically × 20 pixels horizontally. The computing unit 44 detects an approximate motion vector using these reduced images. Subsequently, the calculation unit 44 acquires the encoding target macroblock and the original image of the search area from the frame memory 28, and detects the motion vector while referring to the approximate motion vector.

  When a motion vector is detected, as shown in FIG. 3B, the encoding target macroblock is moved to the right and the motion vector is detected in the same manner. When a motion vector is detected up to the rightmost macroblock in the uppermost stage of the input image, the next macroblock is moved to. In this way, motion vectors of all macroblocks of the input image are detected.

  4 and 5 are diagrams for explaining a motion vector detection procedure according to the present embodiment. FIG. 4A shows a search area when detecting the motion vector of the macro block at the upper left corner of the input image. As in the case of FIG. 3A, a reduced image of the search area 64a is generated to detect an approximate motion vector, and then a motion vector is detected using the original image. At this time, the reduced image of the search area 64 a generated by the reduced image generation unit 40 is stored in the reduced image holding unit 42.

  In this example, the reduced image holding unit 42 is configured by a memory having a capacity enough to store reduced images of all search areas 64 used for detecting motion vectors of all macroblocks of an input image of one frame. Is done. That is, it is configured by a memory having a capacity capable of storing a reduced image of the reference image for about one frame. When detecting a motion vector of a macroblock near the edge of an input image, as shown in FIG. 4A and the like, if a search region is set so as to protrude from the reference image, the reference image is correspondingly increased. A memory having a larger capacity than one frame of the reduced image may be provided.

  Next, as shown in FIG. 4B, the motion vector detection of the right macroblock is performed. When the reduced image of the search area 64b is generated, the reduced image of the area 66b that overlaps the search area 64a has already been generated and stored in the reduced image holding section 42, and therefore can be read from the reduced image holding section 42. That's fine. Therefore, the reduced image generation unit 40 may read out only the pixel data of the area 68b that does not overlap the search area 64a from the frame memory 28 and generate a reduced image. At this time, the newly generated reduced image of the area 68 b is also stored in the reduced image holding unit 42.

  As shown in FIG. 4C, when detecting the motion vector of the subsequent macroblock, it is not necessary to newly generate all the reduced images in the search region 64c, and the reduced image in the region 66c. May be read from the reduced image holding unit 42 and only the reduced image in the region 68c may be generated.

  FIG. 5 is a diagram for explaining the procedure for detecting the motion vectors of the second and subsequent macroblocks of the input image. As shown in FIG. 5A, when detecting the motion vector of the leftmost macroblock in the second row, the reduced image in the region 66d in the search region 64d may be read from the reduced image holding unit 42. It is only necessary to generate a 68d reduced image.

  As shown in FIGS. 5 (b) and 5 (c), in the macroblocks in the second and subsequent columns of the second stage, a new reduced image is generated in the lower right area of the search areas 64e and 64f. Only 68e and 68f are sufficient.

  As described above, by providing the reduced image holding unit 42, when the calculation unit 44 detects the approximate motion vector using the reduced image, the reference memory is searched from the frame memory 28 every time the macroblock to be encoded changes. Rather than acquiring an original image of a region and newly generating a reduced image, a reduced image of a reference image that has already been generated and held in the detection of past approximate motion vectors is read out from the reduced image holding unit 42 and regenerated. Can be used. Thereby, memory access to the frame memory 28 can be drastically reduced, and the processing speed of motion vector detection can be improved. As a result, motion vector detection accuracy can be improved.

  Compare the memory transfer amount between the method shown in FIG. 3 and the method shown in FIGS. In the example of FIG. 3, since the original image of the search area 64 is read from the frame memory 28 for each macroblock, it is necessary to transfer pixel data for 80 × 80 = 6400 pixels. On the other hand, in the examples of FIGS. 4 and 5, the macroblock in the first uppermost row is the same as the example in FIG. 3, but in the macroblocks in the second and subsequent rows of the uppermost row, memory transfer is performed. The amount is reduced. For example, in the macroblock in the uppermost second column, as shown in FIG. 4B, since the area 68b reads the original image from the frame memory 28, 80 × 16 = 1280 pixels, and the area 66b is a reduced image holding unit. Since the reduced image is read out from 42, 20 × 16 = 320 pixels are read out. Since the newly generated reduced image of the area 68b is stored in the reduced image holding unit 42, 20 × 4 = 80 pixels are written. That is, pixel data for a total of 1680 pixels is transferred. This corresponds to 26% of the example shown in FIG.

  Further, in the second and subsequent macroblocks in the second row, as shown in FIG. 5B, since the original image is read from the frame memory 28 in the area 68e, the area 66e is reduced by 16 × 16 = 256 pixels. Since the reduced image is read out from the image holding unit 42, 20 × 16 + 16 × 4 = 384 pixels are read out. Then, since the newly generated reduced image of the area 68e is stored in the reduced image holding unit 42, 4 × 4 = 16 pixels are written. That is, pixel data for a total of 656 pixels is transferred. This corresponds to 10% of the example shown in FIG.

  The reduced image holding unit 42 may hold the pixel data in an arrangement similar to the arrangement of the pixels of the reduced image. As a result, reading and writing can be easily controlled, and reading and writing can be performed at high speed.

  FIG. 6 is a diagram for explaining another example of the motion vector detection procedure of the present embodiment. In this example, not all of the reduced images in the search area 64 are stored in the reduced image holding unit 42, but only the area that overlaps the search area used in the next motion vector detection is stored in the reduced image holding unit 42. Keep it. As shown in FIG. 6A, a reduced image of the search area 64a is generated when detecting the motion vector of the upper left macroblock. At this time, as shown in FIG. 6D, the reduced image in the area 69a that overlaps the search area 64b used in the motion vector detection of the next right macroblock to be performed next is stored in the reduced image holding unit.

  As shown in FIG. 6B, when detecting the motion vector of the macroblock in the uppermost second row, the area 66b of the search area 64b overlaps the previous search area 64a. A reduced image may be read from the holding unit 42. In an area 68b that does not overlap the previous search area 64a, pixel data of the original image is read from the frame memory 28, and a new reduced image is generated. At this time, as shown in FIG. 6E, the reduced image holding unit 42 is overwritten with the reduced image in the region 69b that overlaps the search region 64c used in the next motion vector detection of the next right macroblock.

  The memory transfer amount in this example is evaluated. In the macroblocks in the second and subsequent columns of each stage, as shown in FIG. 6B, the area 68b reads the original image from the frame memory 28, so 80 × 16 = 1280 pixels, and the area 66b is a reduced image holding unit. Since the reduced image is read out from 42, 20 × 16 = 320 pixels are read out. Since the reduced image in the area 69b is stored in the reduced image holding unit 42, 20 × 16 = 320 pixels are written. That is, pixel data for a total of 1920 pixels is transferred. This corresponds to 30% of the example shown in FIG.

  In this example, a memory having a capacity capable of holding the reduced image in the area 69 may be provided as the reduced image holding unit 42, so that the memory transfer amount can be reduced while suppressing an increase in cost, size, weight, power consumption, and the like. Effect is obtained.

  Any of the above-described techniques can reduce access to the frame memory 28 and encode a high-resolution image at high speed.

  The present invention has been described based on the embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  In the embodiment, the reduced image is stored in the reduced image holding unit 42 in the same arrangement as the pixel arrangement of the reduced image. However, the newly generated reduced image in the area 68 is sequentially overwritten on the unused area. May be. Thereby, the capacity | capacitance of the required reduced image holding | maintenance part 42 can be made small. Further, not only the reference image but also the input image may be stored in the reduced image holding unit 42 and reused similarly. As a result, memory access can be further reduced and the processing speed can be increased.

  In the embodiment, the reduced image generation unit 40 generates the reduced image of the input image and the reference image. However, the reduced image of the input image may be generated by another configuration and input to the motion vector detection circuit 24.

It is a figure which shows the whole structure of the image coding apparatus which concerns on embodiment of this invention. It is a figure which shows the internal structure of a motion vector detection circuit. It is a figure for demonstrating the detection procedure of the conventional motion vector. It is a figure for demonstrating the detection procedure of the motion vector of this Embodiment. It is a figure for demonstrating the detection procedure of the motion vector of this Embodiment. It is a figure for demonstrating another example of the detection procedure of the motion vector of this Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Image coding apparatus, 24 motion vector detection circuit, 26 motion compensation circuit, 28 frame memory, 30 encoding circuit, 32 decoding circuit, 34 output buffer, 36 code amount control circuit, 38 reference mode selection circuit, 40 reduced image Generating unit, 42 reduced image holding unit, 44 calculating unit;

Claims (1)

A motion vector detection device that detects a motion vector between a first image and a second image that is a reference image of the first image,
A reduced image generation unit that acquires the second image and generates a reduced image having a lower resolution than the original image;
A reduced image holding unit for storing at least a part of the reduced image;
A calculation unit for calculating a matching between images and detecting a motion vector,
The computing unit refers to a motion vector obtained by computing matching between the reduced image of the first image and the reduced image of the second image in units of macroblocks of a predetermined size, and has an original resolution. A motion vector is detected by calculating a matching between the first image and the second image;
When detecting a motion vector using the reduced image, a portion of the reduced image held in the reduced image holding unit is read from the reduced image holding unit and held in the reduced image holding unit. The part that has not been generated is generated by the reduced image generation unit,
The reduced image holding unit includes a memory capable of holding the entire reduced image of the second image, and stores pixel data in the memory in an arrangement similar to the arrangement of pixels of the reduced image. motion-out vector detecting device you wherein a.

Images