WO2008020470A1 - Decoding method and device - Google Patents

Abstract

A decoding method in which a plurality of processing units carry out processing per macro block for subject coded image data allocates a plurality of macro block lines to a processing range of which each processing unit is in charge and each processing unit carries out the processing of a notable macro block by using a processed result of a reference macro block of the last macro block line prior by one to a macro block line to which the notable macro block belongs. By using a processed result of a reference macro block of the last macro block line in the processing range of the first processing unit, the processing is carried out for the notable macro block of the first micro block in the processing range of the second processing unit.

Description

 Specification

 Decoding method and apparatus

 Technical field

 TECHNICAL FIELD [0001] The present invention relates to a decoding method and apparatus, and more particularly to a decoding method and apparatus for decoding moving picture data encoded (ie, compressed) in accordance with a moving picture coding standard such as H.264. Technology

 [0002] H. 264 or MPEG (Moving Picture Experts Group) 4—AVC (Advanced Video Coding) is developed by JVT, a joint standardization organization between ISO and ITU—T (ISO / I EC 14496-10 or ITU — T Q6Z16), a new compression / decompression technology standardized in 2003. The H.264—AVC system (hereinafter simply referred to as the H.264 system) is known to be used in broadcasting for mobile terminals, which is called “one-segment broadcasting” in digital terrestrial television broadcasting. In H.264 format, 4 x 4 pixel integer conversion, 9-direction intra prediction, 7 types of sub macro block division, minimum 4 x 4 pixel motion vector, multi-frame reference New technologies such as in-loop filters and arithmetic codes have been introduced to achieve a high compression ratio. By using the H.264 system, it is said that the same playback image quality can be obtained with half the stream size compared to the MPEG 2 system.

 However, the algorithm used in the H.264 method has a large amount of processing because it places importance on code efficiency. For this reason, signal processing devices that perform H.264 encoding and decoding (encoding and decoding) include a configuration that performs parallel processing by software and a configuration that performs parallel processing by dedicated hardware. Has been used.

[0003] When performing image processing by parallel processing using software or hardware, there are a method of assigning processing in units of images and a method of assigning in units of processing. In the case of hardware implementation, a method of allocating in units of processing is often adopted. In the case of parallel processing by software, both methods can be used. When the method of allocating processing in units of images is adopted, the number of processors with less processing bias between processors is increased. Expansion to a large screen is facilitated by making it calorie.

 [0004] However, each process of the H.264 method has a complicated data dependency, and a lot of synchronous processing is required when performing parallel processing.

[0005] An H. 264 encoding / decoding method is proposed in Patent Document 1, for example.

. Also, a method power for performing H.264 decoding decoding by parallel processing is proposed in Non-Patent Document 1, for example.

 Patent Document 1: Japanese Patent Laid-Open No. 2005-354361

 Non-Special Reference 1: Yen-Kuang hen et al., Implementation of H. 2D4 Encoder and De coder on Personal Computers ", Special Issue on Emerging H.264 / AVC Video Coding Standard, Journal of Visual Communication and Image Representation, Kumar, M andal and Panchanathan

 Disclosure of the invention

 Problems to be solved by the invention

[0006] The H.264 method has a limitation on MBs that can be operated in parallel in parallel processing in which data dependence is large between macro blocks (MB). For example, in the H.264 decoding process, there is a process that requires MB data on the left, upper left, upper, and upper right of the target MB in MB units. In such a process, in order to start processing the target MB, it is necessary to complete the processing of the left, upper left, upper, and upper right MBs of the target MB and transfer the data. For this reason, when MB processing as described above is performed by software processing of multiple processors or hardware processing of a decoder, synchronization processing such as waiting for processing completion and data transfer frequently occurs, resulting in a large overhead. There was a problem of trying.

 [0007] Therefore, the present invention provides a decoding that can perform efficient and efficient processing for image processing that requires processing with complicated data dependency, such as decoding processing in H.264. The general purpose is to provide a method and apparatus.

 Means for solving the problem

[0008] The above problem is a decoding method in which a plurality of processing units perform parallel processing in units of macroblocks on encoded image data to be processed, and each processing unit performs processing. A plurality of macro block lines are assigned to the processing range in charge, and processing of the target macro block processed by each processing unit is performed from the macro block line to which the target macro block belongs.

The second processing unit uses the processing result of the reference macroblock of the last macroblock line within the processing range of the first processing unit using the processing result of the reference macroblock of the preceding macroblock line. This can be achieved by a decoding method characterized by processing the target macroblock of the first macroblock line within the processing range.

 [0009] The above-described problem includes a plurality of processing units that perform parallel processing of encoded image data to be processed in units of macroblocks, and a plurality of macroblocks are included in a processing range in which each processing unit is responsible for processing. The processing of the target macroblock to which each processing unit processes is assigned using the processing result of the reference macroblock of the macroblock line that precedes the macroblock line to which the target macroblock belongs. The target macroblock of the first macroblock line in the processing range of the second processing unit is processed using the processing result of the reference macroblock of the last macroblock line in the processing range of the first processing unit. This can be achieved by a decoding device characterized by the following.

 The invention's effect

 [0010] According to the present invention, there is provided a decoding method and apparatus capable of performing efficient processing for image processing that requires processing having complicated data dependency, such as H.264 decoding processing. Can be realized.

 Brief Description of Drawings

FIG. 1 is a diagram for explaining data dependence of H.264 decoding process.

 FIG. 2 is a diagram for explaining a possible processing order and data transfer when a processor is allocated for each MB.

 FIG. 3 is a diagram for explaining the processing order and data transfer of the present invention when a processor is allocated for each MB.

 FIG. 4 is a diagram for explaining the processing order and data transfer of the present invention when a processor is allocated for each MB.

FIG. 5 is a diagram for explaining the processing sequence and data transfer of the first embodiment when two processors are used. FIG. 6 is a flowchart for explaining the outline of processing of the first embodiment.

 FIG. 7 is a diagram showing the overhead of the first embodiment in comparison with a comparative example.

 FIG. 8 is a diagram for explaining the processing sequence and data transfer of the first embodiment when three processors are used.

 FIG. 9 is a flowchart illustrating the processing of three processors in the first embodiment.

 FIG. 10 is a diagram for explaining the processing sequence and data transfer of the second embodiment when two processors are used.

 FIG. 11 is a flowchart for explaining the outline of processing of the second embodiment.

 FIG. 12 is a diagram for explaining the processing sequence and data transfer of the second embodiment when three processors are used.

 FIG. 13 is a flowchart for explaining processing of three processors in the second embodiment. Explanation of symbols

[0012] 11, 21 processor

 BEST MODE FOR CARRYING OUT THE INVENTION

 [0013] The H.264 method has a limitation on MBs that can be operated in parallel in parallel processing in which data dependence is large between macro blocks (MB). The H.264 method of intra prediction, motion vector (MV) prediction, and deblocking filter (DF) processing depend on the data that requires MB data around the target MB. There is sex. For this reason, synchronous processing, such as processing completion waiting and data transfer, occurs frequently. There is also a limit to the number of MBs that can be calculated in parallel.

 [0014] Therefore, the present invention provides a parallel processing order focusing on the locality of the encoded image data (moving image data) and simplification of the synchronization processing within this limitation.

[0015] In the decoding process of the H.264 method, there is a process in units of MB, and processing that requires reference MB data on the left, upper left, upper, and upper right of the target MB indicated by hatching as shown in FIG. is there. Fig. 1 is a diagram for explaining the data dependence of the decoding processing of the H.264 method. (A) shows the data dependence in the case of intra prediction and MV prediction, and (b) shows the data dependence in the case of DF processing. Indicates dependency. In Figure 1, each rectangular area represents one MB. As can be seen from Fig. 1, in order to start the processing of the target MB, in the case of intra prediction or MV prediction, It is necessary to complete the processing of the reference MB on the left, upper left, upper, and upper right and transfer the data. In the case of DF processing, the processing of the reference MB above the target pixel is completed and the data is transferred. It will be necessary to deliver

[0016] For this reason, when the decoding apparatus performs MB processing as described above by software processing of a plurality of processors or hardware processing of a decoder, it may be performed in the processing order as shown in FIG. 2, for example. . FIG. 2 is a diagram illustrating a possible processing order and data transfer in which a processor (or decoder) is allocated for each MB in the case of intra prediction as an example. In Fig. 2, each rectangular area indicates one MB, thick solid arrows indicate processing order, thin solid arrows indicate data transfer, and two processors # 1 and # 2 are used for convenience of explanation. To do. However, if this possible processing order is used, the processors # 1 and # 2 that perform processing change for each macroblock line (MB line), and synchronization processing such as waiting for processing completion and data transfer is performed. It occurs frequently and increases the overhead.

 [0017] Therefore, in the present invention, the decoding apparatus for the encoded image data (moving image data) to be processed is software processing of a plurality of processors, or hardware processing of a plurality of decoders. When performing processing in units of MB as described above with a plurality of processing units, the range in which each processor (or decoder) is in charge of processing (hereinafter also referred to as the processing range) is not for each MB line but for each MB line. Then, processing is performed in an order that gives priority to MBs arranged in a substantially vertical direction as shown in FIG. 3 or FIG. 3 and 4 are diagrams for explaining the processing order and data transfer according to the present invention in which a processor is allocated for each MB in the case of intra prediction. In Fig. 3 and Fig. 4, each rectangular area indicates one MB, thick solid arrows indicate processing order, and thin solid arrows indicate data transfer. For convenience of explanation, two processors # 1, # 2 are used in FIG. 3, three processors # 1- # 3 are used in FIG. 4, and each processor # 1, # 2, # 3 is in charge of processing. The processing range shall be every 4MB line.

[0018] According to the present invention, the boundary of the processing range in charge between different processors or different decoders is reduced, and an increase in overhead due to synchronization processing can be suppressed. For example, as shown in Figure 3, with the two processors # 1 and # 2, When processing, assuming that the number of MB lines in the processing range assigned to each processor # 1, # 2 is N, the number of synchronization processing can be reduced to 1ZN times compared to the above conceivable method. Before the first synchronization processing, for example, one processor # 1 (N + DNZ requires 2 MB processing, while the other processor # 2 cannot start processing, so the synchronization overhead is OH and decryption processing When the total number of MBs subject to TM is TM and the processing amount per MB is A, when the condition OH XTM X (N-1) / N> AX (N + 1) X NZ2 holds, The effect of reducing the synchronization overhead OH by introducing the invention can be obtained.

 [0019] It should be noted that the processing of the present invention in which a processor is allocated for each MB in the case of DB processing is also arranged in a substantially vertical direction (that is, a direction other than the horizontal direction on the same MB line) as in the case of the intra prediction. The same effects as described above can be obtained in the order in which the given MBs are prioritized. Details thereof will be described together with the following examples.

 Hereinafter, embodiments of the decoding method and apparatus of the present invention will be described with reference to the drawings.

 Example

 [0021] (First embodiment)

 First, a first embodiment of the decoding method and apparatus according to the present invention will be described with reference to FIGS.

 FIG. 5 is a diagram for explaining the processing order and data transfer of the first embodiment when two processors are used. In Fig. 5, each rectangular area represents one MB, the numbers in each rectangular area indicate the processing order, and the thin solid arrows indicate data transfer. For convenience of explanation, it is assumed in FIG. 5 that two processors # 1 and # 2 are used, and each processor # 1 and # 2 is in charge of processing every 3 MB line.

 [0023] When processing the target MB in one frame of the encoded image data (movie data) to be processed, such as intra prediction and MV prediction, the reference MB including the upper right of the target MB When data is required, processing is performed in the order shown in Fig. 5. Each MB line is composed of, for example, 5 MB, and each MB is composed of, for example, data of 16 × 16 pixels.

[0024] When the processing of the target MB is completed, the processing of the reference MB at the lower left is performed next. When processing reaches the attention MB of the last (bottom or bottom) MB line of the processor's processing range, the top (first) MB line of this processing range is unprocessed MB That is, the leftmost MB is processed as the target MB. If there is no outstanding MB in the first MB line, look for the candidate for the target MB from the second MB line from the top, and if there is no candidate in the second MB line, the third below The MB line power also looks for candidates. Since each processor # 1, # 2 or each dedicated hardware (decoder) has 3 MB lines in the processing range, 3 (3 + 1) / 2 = 6 MB After that, repeat the process of performing one synchronization process for 3 MB processes until the end of the processing range (the rightmost MB of the bottom MB line) is reached. . In this way, the process is repeated until all MBs within each processing range have been processed.

 [0025] That is, as shown in FIG. 5, when the processing of the leftmost noticed MB “1” in the first MB line is completed, processor # 1 is left unprocessed in the first MB line and leftmost. Processing with MB “2” as the target MB. Next, MB “3” in the second MB line at the lower left is processed as the target MB and the leftmost MB “4” in the first MB line is processed as the target MB. I do. Next, the MBs “5” and “6” in the second and third MB lines on the lower left are treated as MBs of attention, and 6 MBs are processed. After that, the process of performing one synchronization process for three MB processes is repeated until the end of the processing range (the rightmost MB of the bottom MB line) is reached.

[0026] On the other hand, the processor # 2 performs processing in the same processing order as the processor # 1, but the first MB line to be processed first is transferred by synchronous processing. Use the processing result of processor # 1. In other words, when processor # 1 has processed the first 6 MBs and the next 3 MBs, processor # 2 has MBs “6” and “9” of each third MB line. In the MB line processed by processor # 2 using the processing result of "", the leftmost attention MB "1" of the first MB line is processed. When processing of the leftmost noticed MB “1” of the first MB line is completed, when processor # 1 has processed the next three MBs, processor # 2 is the third MB. In the MB line processed by processor # 2 using the processing result of MB “12” in the line, processing is performed with the left MB “2” being the unprocessed first MB line as the target MB. After that, processor # 2 uses the same processing order as processor # 1 above. For the MB line processed by processor # 2, the processing result of processor # 1 is used for the processing of the first MB line. Processing Repeat until all MBs in the range have been processed.

[0027] If the number of MB lines in each processor # 1, # 2 or each dedicated hardware (decoder) in the processing range is N, N (N + 1) Z2 MBs are processed first. Then, repeat the process of N MBs and one synchronization process until the right end of the processing range is reached. As a result, in the repetitive part of the processing range, one synchronization process is required for every N MBs, and the synchronization process is reduced to 1 / N compared to the above conceivable method (hereinafter referred to as a comparative example). Can be reduced

[0028] When the overhead of synchronization processing is OH, in this embodiment, as shown in Fig. 7 (a), the number of times of synchronization processing is 1 / N compared to the comparative example of Fig. 7 (b). become. FIG. 7 is a diagram showing the overhead OH of the first embodiment in comparison with the comparative example. In Fig. 7, the rectangular area that is marked with no and tatch indicates MB processing, and the rectangular area that is painted black indicates synchronization processing. In this example, processor # 2 can start processing when it is delayed by {N (N + 1) Z2 + N} MB of processing since processor # 1 started processing. However, it can be seen that when the image size is large and the overhead OH of the synchronization processing is large, the processing can be performed more efficiently than the comparative example.

 FIG. 6 is a flowchart for explaining the outline of the processing of the first embodiment. The processing shown in FIG. 6 is executed by the processor 11. The processor 11 is an information processing apparatus having a known configuration including a CPU, a storage unit, and the like, and the decoding apparatus has a configuration in which a plurality of such processors 11 are connected so as to be capable of parallel processing.

 [0030] In FIG. 6, step S1 processes the first N (N + 1) Z2 MBs. As a result, the processing of MBs “1” to “6” of the first to third MB lines shown in FIG. 5 is performed. In step S2, the processes in steps S3 and S4 are repeated until the rightmost MB of each MB line in the processing range of one frame of the encoded image data to be processed is processed. In step S3, N MBs are processed, and in step S4, synchronous processing including waiting for completion of processing and data transfer is performed. In step S5, it is determined whether or not steps S2 to S4 have been performed for the last MB line in the processing range. If the determination result is NO, the process returns to step S2, and if YES, the process ends.

[0031] When two processors # 1, # 2 are used for parallel processing as described above, each processor # 1, # 2 # 2 performs the processing shown in Figure 6. However, in the case of processor # 2, the processing for processor MB is the same as the processing for processor # 1 in that reference MB data including the upper right of the target MB is required. Force that is the same The MB line processed by processor # 2 uses the processing result for the MB of the third MB line in the MB line processed by processor # 1 when processing the MB of the first MB line Only different.

FIG. 8 is a diagram for explaining the processing sequence and data transfer in the first embodiment when three processors are used, and FIG. 9 explains the processing of the three processors in the first embodiment. It is a flowchart. In Fig. 8, each rectangular area indicates one MB, the numbers in each rectangular area indicate the processing order from the upper left to the upper right and the lower right of the processing range, and the thin solid arrows indicate data transfer. Also, for convenience of explanation, it is assumed that the processing range in which each processor # 1, # 2, # 3 is in charge of processing is N macroblock (MB) lines, and each MB line is composed of W MBs. Here, N and W are both integers of 2 or more, and N ≠ W or N = W, but preferably N <W. Each MB is composed of data of 16 × 16 pixels, for example.

 [0033] When processing the target MB in one frame of the encoded image data to be processed, such as intra prediction and MV prediction, reference MB data including the upper right of the target MB is required. In this case, processing is performed in the order shown in FIG.

 [0034] FIG. 9 shows the processing of three processors # 1, # 2, and # 3. Processor # 1 performs steps S101—1 to S108—1! /, Processor # 2ί steps S101—2 to S108—2, S11 1—2 to S113—lines 2! /, Processor # 3ί Steps S101-3 to S108-3 and Sill-3 to S113-3 are performed. In FIG. 9, substantially the same steps are indicated by subscripts “1” to “−3” corresponding to the processors # 1 to # 3 to the same reference numerals.

[0035] First, the processing of processor # 1 will be described. When processing of processor # 1 is started, step S101—1 sets variable i (integer) to i = 1. In step S102-1, the i-th MB "i" is processed !, and in step S103-1, it is determined whether the MB is located at the left end of the processing range! /. If the decision result in the step S103-1 is NO, a step S104-1 decides whether or not the MB is located at the lower end of the processing range. If the determination result of step S104-1 is NO, step S105-1 will process the lower left MB of the current MB and process Returns to step S103—1. On the other hand, if the decision result in the step S104-1 is YES, the step S106-1 performs data transfer to the processor # 2, and the step S107-1 determines whether i = (W + N-1). Determine. In the data transfer in step S106—1, the MB processing result of the Nth MB line is transferred to the processor # 2. If the decision result in the step S107-1 is NO, the step S108-1 increments i to i = i + 1, and the process returns to the step S102-1. If the decision result in the step S107-1 is YES, the process of the processor # 1 is finished.

 Next, the processing of processor # 2 will be described. When the processing of processor # 2 is started, step S101-2 sets variable i (integer) to i = 1. Step S 111-2 performs a synchronization process waiting for data transfer from processor # 1. Therefore, at first, the processing result of the MB of the Nth MB line processed by processor # 1 is required for processor # 2 to process the first MB “1” of the first MB line. When transferred from # 1, the process proceeds to step S112-2. In step S112-2, it is determined whether or not i≤W. If the determination result is YES, step S113-2 performs synchronization processing waiting for data transfer from processor # 1. Therefore, when the processing result of the MB of the Nth MB line processed by processor # 1, which is necessary for processor # 2 to process the MB, is transferred from processor # 1, the process proceeds to step S102. — Go to step 2. If the decision result in the step S112-2 is NO, the process advances to a step S102-2. The other processing of processor # 2 is basically the same as the processing of processor # 1.

[0037] Next, the processing of the processor # 3 will be described. When processing of processor # 3 is started, step S101-3 sets variable i (integer) to i = 1. Step S 111-3 performs a synchronization process waiting for data transfer from processor # 2. Therefore, at first, the processing result of the MB of the Nth MB line processed by processor # 2 is required for processor # 3 to process the first MB “1” of the first MB line. When transferred from # 2, the process proceeds to step S112-3. In step S112-3, it is determined whether or not i≤W. If the determination result is YES, step S113-3 performs synchronous processing waiting for data transfer from processor # 2. Therefore, the processing result of MB of the Nth MB line processed by processor # 2 that is necessary for processor # 3 to process MB is transferred from processor # 2. Then, the process proceeds to step S102-3. If the decision result in the step S112-3 is NO, the process advances to a step S102-3. The other processing of processor # 4 is basically the same as the processing of processor # 2 above.

 (Second embodiment)

 Next, a second embodiment of the decoding method and apparatus according to the present invention will be described with reference to FIGS.

FIG. 9 is a diagram for explaining the processing order and data transfer of the second embodiment when two processors are used. In Fig. 9, each rectangular area represents one MB, the numbers in each rectangular area indicate the processing order, and the thin solid arrows indicate data transfer. For convenience of explanation, it is assumed in FIG. 9 that two processors # 1 and # 2 are used, and the processing range in which each processor # 1 and # 2 is in charge of processing is every 3 MB line.

[0039] When processing the target MB in one frame of the encoded image data to be processed, as in DF processing, when reference MB data above the target MB is required, Processing is performed in the order shown. Each MB line has 4 MB power, for example, and each MB has 16 x 16 pixel data, for example.

[0040] When the processing of the target MB is completed, the next MB processing is performed. When processing reaches the attention MB of the last (bottom) MB line in the processor's processing range, the leftmost MB of the unprocessed MBs in the top (first) MB line of this processing range Process as the MB of interest. Each processor # 1, # 2 or each dedicated hardware (decoder) is responsible for 3 MB lines, so in this way the process of performing 1 synchronization process for 3 MB processing Repeat until the end of the processing range (the rightmost MB of the bottom MB line) is reached. In this way, the process is repeated until all MBs in each processing range have been processed.

[0041] That is, as shown in FIG. 10, when the processing of the leftmost noticed MB “1” in the first MB line is completed, the processor # 1 finishes processing the second MB located under the MB “1”. Processing is performed with the leftmost MB “2” in the MB line as the target MB. Next, processing is performed with the leftmost MB “3” of the third MB line located under MB “2” as the target MB. Next, the left MB “4” in the first MB line that has not been processed is processed as the target MB, and MBs “5” and “6” in the second and third MB lines located below are processed. Processing with the attention MB. After that, do the same Repeat the process of performing one synchronization process for three MB processes until the end of the processing range (the rightmost MB of the bottom MB line) is reached.

[0042] On the other hand, the processor # 2 performs processing in the same processing order as the processor # 1, but the first MB line to be processed first is transferred by synchronous processing. Use the processing result of processor # 1. In other words, when the first three MBs are processed by processor # 1, the processing result of MB “3” in the third MB line is used, and the MB line processed by processor # 2 is 1. The leftmost attention MB “1” of the second MB line is processed. When the processing of the leftmost noticed MB “1” in the first MB line is completed, the MB “2” in the second MB line is processed as the noticed MB. After that, processor # 2 has the same processing order as that of processor # 1 above. For the processing of the first MB line, the processing result of processor # 1 is used to determine all MBs within the processing range. Repeat the process until the process is completed.

 [0043] If the number of MB lines in each processor # 1, # 2 or each dedicated hardware (decoder) is in the processing range is N, the processing range is N MB processing and one synchronous processing. Repeat until the right edge of is reached. As a result, in the repetitive part of the processing range, one synchronization process is required for every N MBs, and the synchronization process can be reduced to 1 / N compared with the comparative example.

 FIG. 11 is a flowchart for explaining the outline of the processing of the second embodiment. The processing shown in FIG. 10 is executed by the processor 21. The processor 21 is an information processing apparatus having a known configuration including a CPU, a storage unit, and the like, and the decoding apparatus has a configuration in which a plurality of such processors 21 are connected so as to be capable of parallel processing.

 In FIG. 11, step S 21 repeats the processing of steps S 22 and S 23 until the rightmost MB of each MB line in the processing range is processed. In step S22, N MBs are processed, and in step S23, synchronous processing including waiting for completion of processing and data transfer is performed. In step S24, it is determined whether or not the power of steps S21 to S23 has been performed on the last MB line in the processing range. If the determination result is NO, the process returns to step S21, and if it is YES, the process ends.

[0046] As described above, when two processors # 1, # 2 are used for parallel processing, each processor # 1, # 2 # 2 performs the processing shown in Fig. 11. However, in the case of processor # 2, when processing for the target MB requires reference MB data including the target MB, processor # 1 This is the same as the processing, except that the MB line processed by processor # 2 uses the processing result for the MB of the third MB line of processor # 1 for the processing of the first MB line.

 FIG. 12 is a diagram for explaining the processing order and data transfer in the second embodiment when three processors are used, and FIG. 13 explains the processing of the three processors in the second embodiment. It is a flowchart. In Fig. 12, each rectangular area indicates one MB, the numbers in each rectangular area indicate the processing order from the upper left to the upper right and lower right of the processing range, and the thin solid arrows indicate data transfer. Also, for convenience of explanation, the processing range in which each processor # 1, # 2, # 3 is in charge of processing is, for example, for each P, Q, R macroblock (MB) line according to each processing performance, Each MB line shall consist of W MBs. Here, P, Q, R, and W are all integers of 2 or more, and P, Q, and R may be equal to or different from N, but are preferably smaller than W. In this embodiment, P, Q, and R satisfy the relationship P> R> Q. 1S It is not limited to this relationship. Each MB is composed of, for example, 16 × 16 pixel data.

 [0048] When processing the target MB in one frame of the encoded image data to be processed as in DF processing, when the reference MB data including the upper part of the target MB is required, the processing shown in FIG. Processing is performed in the order shown in FIG.

 [0049] FIG. 13 shows processing of three processors # 1, # 2, and # 3. Processor # 1 performs steps S201—1 to S207—1! /, Processor # 2ί steps S201—2 to S207—2, S2 11—2! — 3, S211— Perform 3 In FIG. 13, substantially the same steps are indicated by adding the suffixes “1” to “1 3” corresponding to the processors # 1 to # 3 to the same reference numerals.

First, the processing of processor # 1 will be described. When the processing of processor # 1 is started, step S201-1 sets variable i (integer) to i = 1. Step S 202-1 performs processing of the i-th MB “i” !, and step S 203-1 determines whether or not the MB is located at the lower end of the processing range! /. If the judgment result in step S203-1 is NO, step S204-1 The MB under the current MB is processed, and the process returns to step S203-1. On the other hand, if the decision result in the step S203-1 is YES, the step S205-1 performs data transfer to the processor # 2, and the step S206-1 does i = (W + P-1)? Determine whether or not. In step S205—1 data transfer, the MB processing result of the Pth MB line is transferred to processor # 2. If the decision result in the step S206-1 is NO, the step S207-1 increments i to i = i + 1, and the process returns to the step S202-1. If the decision result in the step S206-1 is YES, the process of the processor # 1 is finished.

[0051] Next, the processing of processor # 2 will be described. When the processing of processor # 2 is started, step S201-2 sets variable i (integer) to i = 1. Step S211-2 performs a synchronization process waiting for data transfer from processor # 1. Therefore, at first, the processing result of the MB of the Pth MB line processed by processor # 1 is required for processor # 2 to process the first MB "1" of the first MB line. When transferred from # 1, processing proceeds to step S202-2. The other processing of processor # 2 is basically the same force S as the processing of processor # 1, and step S206-2 determines whether i = (W + Q—1).

 Next, the processing of processor # 3 will be described. When processing of processor # 3 is started, step S201-3 sets variable i (integer) to i = 1. Step S211-3 performs a synchronous process waiting for data transfer from processor # 2. Therefore, at first, the processing result of the MB of the Qth MB line processed by processor # 2 is required for processor # 3 to process the first MB “1” of the first MB line. When transferred from # 2, the process proceeds to step S202-3. The other processing of the processor # 4 is basically the same as the processing of the processor # 2, and the step S206-3 determines whether i = (W + R—1).

[0053] In each of the above embodiments, the ability of the decoding apparatus to perform the above-described MB unit processing on the encoded image data to be processed by software processing of a plurality of processors. When processing in units of MB as described above by hardware processing of a plurality of decoders, even if a plurality of physically separate decoders are used as the decoder, physically a plurality of decoder units within a single decoder are used. Needless to say, it may be used. Industrial applicability

 [0054] The present invention is applicable to image processing that requires complicated data-dependent processing such as H.264 decoding processing.

As described above, the present invention has been described with reference to the embodiments. Needless to say, the present invention is not limited to the above embodiments, and various modifications and improvements can be made within the scope of the present invention.

Claims

The scope of the claims  [1] A decoding method in which a plurality of processing units perform parallel processing in units of macroblocks on encoded image data to be processed.  A plurality of macro block lines are assigned to the processing range in which each processing unit is in charge of processing, and the processing of the target macro block processed by each processing unit is performed one macro block preceding the macro block line to which the target macro block belongs. Using the processing result of the reference macroblock of the line,  Using the processing result of the reference macroblock of the last macroblock line within the processing range of the first processing unit, the target macroblock of the first macroblock line within the processing range of the second processing unit is processed. A decoding method, characterized in that:  2. The decoding method according to claim 1, wherein the processing of the plurality of processing units is performed by software processing of a plurality of processors or hardware processing of a plurality of decoders.  [3] The processing of each processing unit is intra prediction or motion vector prediction compliant with the H.264- AVC standard.  3. The decoding method according to claim 1, wherein the reference macroblock is located at the upper right of the corresponding macroblock of interest. 4. The decoding method according to claim 3, wherein the processing ranges of the first and second processing units are the same. [5] The processing of each processing unit is deblocking filtering based on the H.264—AVC standard.  The decoding method according to claim 1 or 2, wherein the reference macroblock is located above the corresponding target macroblock. 6. The decoding method according to claim 5, wherein the processing ranges of the first and second processing units are different.  [7] Using the processing result of the reference macroblock of the last macroblock line in the processing range of the second processing unit, the attention macroblock of the first macroblock line in the processing range of the third processing unit Process, The processing ranges of the first, second, and third processing units are different from each other. The decoding method according to claim 5 or 6. 8. The decoding method according to any one of claims 1 to 7, wherein each processing unit performs a synchronization process every time a predetermined number of macro blocks are processed. [9] The synchronization processing includes waiting for the completion of processing and data transfer, and uses the processing result of the reference macroblock of the first processing unit for processing of the target macroblock of the second processing unit. 9. The decryption method according to claim 8, wherein the decryption method is transferred to the second processing unit.  [10] Provided with multiple processing units that process the encoded image data to be processed in macroblock units in parallel,  Multiple macroblock lines are assigned to the processing range in which each processing unit is in charge of processing.  The processing of the target macroblock processed by each processing unit is performed using the processing result of the reference macroblock of the macroblock line that precedes the macroblock line to which the target macroblock belongs,  Using the processing result of the reference macroblock in the last macroblock line within the processing range of the first processing unit, the target macroblock of the first macroblock line within the processing range of the second processing unit is processed. Decoding device, characterized in that it is performed.  [11] The decoding device according to [10], wherein the plurality of processing units include a plurality of processors that perform software processing or a plurality of decoders that perform hard- er processing.  [12] The processing of each processing unit is intra prediction or motion vector prediction compliant with the H.264—AVC standard.  12. The decoding device according to claim 10 or 11, wherein the reference macroblock is located at the upper right of the corresponding macroblock of interest. 13. The decoding device according to claim 12, wherein the processing ranges of the first and second processing units are the same. [14] The processing of each processing unit is deblocking filter processing compliant with the H. 264— AVC standard. The reference macroblock is located above the corresponding target macroblock 12. The decoding device according to claim 10 or 11. 15. The decoding device according to claim 14, wherein the processing ranges of the first and second processing units are different.  [16] Using the processing result of the reference macroblock in the last macroblock line within the processing range of the second processing unit, the target macroblock of the first macroblock line within the processing range of the third processing unit is used. Process,  The processing ranges of the first, second, and third processing units are different from each other. 16. The decoding device according to claim 14 or 15. 17. The decoding apparatus according to any one of claims 10 to 16, wherein each processing unit performs a synchronization process each time a predetermined number of macro blocks are processed. [18] The synchronization processing includes waiting for the completion of processing and data transfer, and the processing result of the reference macroblock of the first processing unit is used for processing of the target macroblock of the second processing unit. 18. The decoding device according to claim 17, wherein the decoding device transfers to the second processing unit.