[go: up one dir, main page]

WO2006004065A1 - Système de codage d’image vidéo, système de codage ou décodage d’image vidéo, procédé de codage d’image video, et procédé de codage ou décodage d’image video - Google Patents

Système de codage d’image vidéo, système de codage ou décodage d’image vidéo, procédé de codage d’image video, et procédé de codage ou décodage d’image video Download PDF

Info

Publication number
WO2006004065A1
WO2006004065A1 PCT/JP2005/012250 JP2005012250W WO2006004065A1 WO 2006004065 A1 WO2006004065 A1 WO 2006004065A1 JP 2005012250 W JP2005012250 W JP 2005012250W WO 2006004065 A1 WO2006004065 A1 WO 2006004065A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
encoding
decoding
amount
moving image
Prior art date
Application number
PCT/JP2005/012250
Other languages
English (en)
Japanese (ja)
Other versions
WO2006004065A8 (fr
Inventor
Miwako Kanamori
Kentaro Kawakami
Yasuhiro Morita
Hideo Ohira
Masahiko Yoshimoto
Original Assignee
Kanazawa University Technology Licensing Organization Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kanazawa University Technology Licensing Organization Ltd. filed Critical Kanazawa University Technology Licensing Organization Ltd.
Priority to JP2006528865A priority Critical patent/JPWO2006004065A1/ja
Publication of WO2006004065A1 publication Critical patent/WO2006004065A1/fr
Publication of WO2006004065A8 publication Critical patent/WO2006004065A8/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation

Definitions

  • Moving picture coding processing system moving picture coding or decoding processing system, moving picture coding processing method, and moving picture coding or decoding processing method
  • the present invention uses a processor that can change the operating frequency, the operating power supply voltage and the Z or substrate bias voltage, and sequentially encodes a moving image composed of a plurality of frames in units of frames.
  • the present invention relates to a moving image encoding processing system, a moving image encoding processing or decoding processing system, a moving image encoding processing method, and a moving image encoding or decoding processing method.
  • Non-Patent Document 1 Proceedings of IEEE International Symposium on Circuits and System 2001 (May, 2001) pp918-921 "An LSI for Vdd- Hopping and MPEG4 System Based on the Chip, (H. Kawaguchi, G. Zhang , S. Lee, and T. Sakurai)
  • FIG. 22 is a diagram showing a conventional technique for reducing power consumption for the moving image (moving image) encoding processing system shown in Non-Patent Document 1. Note that the same means for reducing power consumption is applied to the moving picture decoding processing system.
  • Non-Patent Document 1 an operation power supply for reducing power consumption when processing moving image coding (particularly MPEG) on a processor that can dynamically change the operation power supply voltage and operation frequency. Demonstrate how to control voltage and operating frequency!
  • Non-Patent Document 1 discloses that, as shown in FIG. 23, when moving picture encoding is performed, calculation of moving picture code key or decoding key is performed in units of frames depending on the intensity of motion in the moving picture. Focusing on the difference in amount, the operating frequency and operating power supply voltage of the processor are controlled to reduce power consumption.
  • the processing time of one frame is limited to the time Tf by the definition of the encoding method (MPEG, etc.), and the encoding process of one frame is completed within the processing time Tf. That power S is needed.
  • R be the number of moving image blocks processed in one time slot Tslot (moving image is processed in units of blocks) (ie, RXN is the number of blocks in one frame), and (RX i) TaccG + l) is the time spent in block processing (that is, the time actually processed for the block group to be processed from time slot Tslotl to time slot Tslot). Trd is the time until the operating power supply voltage and operating frequency stabilize when the voltage is changed. Note that the real time slot RTsloti is a time slot. G Indicates the actual processing time required for processing to be completed in Tsloti. In FIG.
  • the clock frequency Fmax that can be processed sufficiently in the time slots Tsloti and Tslot2 even when the load is maximum. Operate with. If the time Tacc 3 that was powerful in the processing is Tacc3 (Tf—TR2), that is, if the assigned block group has completed processing in the time slots Tslotl and Tslot2, it is assigned to the next time slot Tslot3.
  • Tacc3 Tf—Tacc3—TR3-Trd
  • the processing of the block group assigned to Tslot3 should be completed within this processing time Ttar3.
  • the block group is operated at a lower operating frequency.
  • the processing times Tfl, T12, and ⁇ in Fig. 22 indicate the processing times when operating at each operating frequency Fl, F2, F3 when the load is maximum in time slot Tslot3.
  • the minimum operation frequency can be selected from among the operation frequencies that can process a predetermined number of block groups within a predetermined time. Lowering the operating frequency and operating power supply voltage, and controlling the voltage according to the required processing, lower power consumption.
  • Ff KfZTf
  • Non-Patent Document 1 the operating power supply voltage and the operating frequency are changed a maximum of N times within one frame even though the unit of synchronization of the processing time Tf is one frame. Therefore, low power consumption was not enough.
  • the low power consumption of video encoding or decoding processing in a processor that can control the operating power supply voltage and operating frequency in multiple stages as in the conventional example is achieved by operating power supply many times during the processing of one frame. It was necessary to change the voltage and operating frequency.
  • the unit of processing time constraint is a frame, it is preferable to control at the minimum fixed frequency that enables processing during processing of one frame. Therefore, in this conventional example in which the power supply voltage and the operating frequency are changed at most N times during processing of one frame, the power consumption cannot be sufficiently reduced.
  • the inventors of the present application completed the invention of Patent Document 1 below.
  • the power consumption is reduced by performing the encoding process or the decoding process while operating the processor 1 at a constant operating power supply voltage / operating frequency.
  • Patent Document 1 Japanese Patent Application 2003—48535
  • the subthreshold leakage current is a minute current that flows when the gate voltage of the MOS transistor formed on the semiconductor substrate is lower than the threshold voltage.
  • the power consumption due to this subthreshold leakage current tends to dominate as MOS transistors become finer, and a video signal that uses a processor in which MOS transistors are integrated on a semiconductor substrate is used. This is one of the factors that hinder the reduction of power consumption in a video code key decoding system or decoding key system.
  • This sub-threshold leakage current is obtained at a constant operating frequency Ff at a constant operating frequency Ff as compared with the case where the operating frequency Ff of the processor is varied many times within the processing time Tf of one frame. It is reduced by operating, and the power consumption of the processor can be reduced.
  • the invention of Non-Patent Document 1 described above is effective even when the unit for synchronizing the processing time Tf is one frame. Regardless, the operating frequency has been changed up to N times within one frame, and not only the operating power supply voltage but also the power of subthreshold leakage current is favorable.
  • MOS transistors it is known that the subthreshold leakage current can be controlled by controlling the substrate bias voltage in the semiconductor region in which the MOS transistor is formed.
  • the present invention calculates the amount of computation required for encoding or decoding (hereinafter referred to as the necessary amount of computation) for each frame, and calculates a constant substrate bias voltage, an operating frequency, or a constant substrate bias voltage.
  • the power consumption can be reduced by performing the encoding / decoding process while operating the processor 1 at the voltage / operating frequency.
  • Patent Document 2 Japanese Patent Application 2003—409641
  • Patent Documents 1 and 2 described above when the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the calculation amount actually required for the encoding or decoding process There was a possibility that the failure of the encoding process or the decoding process could not be completed within the time Te allocated for the processing of one frame.
  • Each patent document 1 and 2 also disclosed failure avoidance means for avoiding the failure phenomenon.
  • several failure avoidance means are conceivable, and it is strongly desired to improve the deterioration of image quality caused by the failure avoidance means. .
  • the present invention is for solving the above-mentioned problems, and while realizing low power consumption by controlling the substrate bias voltage, the operating power supply voltage, and the operating frequency, the failure phenomenon is solved.
  • the present invention proposes a video encoding processing system, a video encoding or decoding processing system, a video encoding processing method, and a video encoding or decoding processing method that can be avoided. Means for solving the problem
  • the moving image encoding processing system of the present invention includes a processor functioning as a moving image encoding means for encoding a moving image composed of a plurality of continuous frames in units of frames. Prepared,
  • Necessary calculation amount calculation means for calculating the necessary calculation amount Kp necessary for encoding one frame, and encoding the necessary calculation amount Kp within the time Te previously allocated to the encoding process of the one frame
  • Operation determining means for determining a processing frequency that can be processed, and the processor is pre-allocated! /, The operation frequency determined by the operation determining means within the time period Te and an operation suitable for the operation frequency.
  • a moving image encoding processing system that performs encoding processing of the one frame by the moving image encoding means while operating at a power supply voltage and a Z or substrate bias voltage;
  • It is characterized by comprising first failure avoiding means for reducing the actual calculation amount of the codeh processing at a predetermined timing.
  • the moving image encoding processing method of the present invention includes a moving image encoding step for encoding a moving image composed of a plurality of consecutive frames using a processor in units of frames, and encoding of one frame.
  • Necessary computation amount calculation step for calculating the necessary computation amount Kp, and operation that can signify the required computation amount Kp within the time Te allocated in advance for the signing processing of the one frame
  • An operation determining step for determining the frequency
  • a first failure avoidance step is provided for reducing an actual calculation amount of the sign key processing at a predetermined timing.
  • the first failure avoiding means and the first failure avoiding step reduce the actual amount of computation by changing a part of the sign process to a processing with a smaller amount of computation.
  • the “encoding process” means the process of the operation determining means ⁇ the operation determining step and the encoding process means ⁇ the process of the encoding process step.
  • the “decoding process” is the operation determining process.
  • Means ⁇ Decoding processing means performed after the processing of the action determination step ⁇ Decoding processing step.
  • the necessary calculation amount Kp necessary for the coding process of one frame to be completed within the time Te is calculated,
  • the operating frequency required for the mouth sensor in the time Te is set as an operating frequency that can be processed, and the minimum operating frequency (or close to the minimum) that allows one frame to be processed in the time Te, and Since the encoding process is performed with the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency, power consumption can be reduced.
  • the required calculation amount Kp calculated by the above-described required calculation amount calculation means is smaller than the actually required calculation amount, the actual calculation amount of the sign ⁇ process is the first failure avoidance means or the first Since it is reduced by 1 failure avoidance step, the failure phenomenon can be avoided.
  • the first failure avoiding means is characterized in that a block having color difference signal information is processed as an invalid block among macroblocks in which the sign is not finished.
  • luminance block a block having luminance signal information
  • color difference block a block having color difference signal information
  • the first failure avoiding means performs an intra-frame encoding process on a macroblock for which the encoding has not ended.
  • the intraframe code processing has a smaller amount of computation than the interframe code processing.
  • the necessary amount of calculation is calculated by the necessary amount of calculation means by performing the intra-frame code processing on the macroblock for which the code processing has not been completed. It can be set to a value smaller than or closer to the required calculation amount Kp. This makes it possible to avoid bankruptcy.
  • the first failure avoiding means performs inter-frame code key processing with a motion vector set to 0 without performing motion vector detection for a macroblock for which the code key has not ended.
  • the amount of computation required for the motion vector detection processing is an element that occupies most of the amount of computation in the encoding processing.
  • the calculation amount is larger than that of the normal interframe code processing that performs the motion vector detection processing.
  • the amount of computation of the actual sign process is smaller than the required amount of computation Kp or close to the required amount of computation Kp. Thereby, it becomes possible to avoid the failure phenomenon.
  • the first failure avoidance means is characterized in that a code step is performed with a larger quantum step size for a macroblock that has not been finished.
  • the number of effective blocks generated in a macroblock is increased by performing the encoding process with the quantum step size being increased for a macroblock for which the code is not finished.
  • the number of effective coefficients is suppressed, and the number of executions of processes performed on effective blocks and effective coefficients (for example, when MPEG4 is used as a moving image encoding method, IDCT processing, IQ processing, VLC processing) is reduced.
  • the actual calculation amount of the encoding process can be made smaller than the required calculation amount Kp or close to the required calculation amount Kp. This makes it possible to avoid the bankruptcy phenomenon.
  • the moving image encoding processing system of the present invention has an effect given to the encoding process of the subsequent frame encoded after the first frame by the first failure avoidance means.
  • a mitigating means for mitigating is provided.
  • the first failure avoidance means described above performs the failure avoidance by changing the processing content of the encoding process that should normally be performed, the first failure avoidance means is executed in one frame. If this happens, the image quality of the subsequent frame will be affected by the processing by the first failure avoidance method.
  • the mitigation processing is performed by the mitigation means that mitigates the influence on the encoding processing of the subsequent frame. Degradation of image quality due to encoding processing can be suppressed.
  • the mitigation means applies a quantization step to the macroblock of the subsequent frame corresponding to the macroblock for which the processing by the first failure avoidance means has been executed in the one frame.
  • the feature is to reduce the size of the tape.
  • the present invention by reducing the quantization step size, it is possible to improve the image quality by increasing the code amount allocated to the corresponding macroblock, and the failure due to the first failure avoiding means. Even when the avoidance process is performed, the influence on the subsequent frame encoded after the one frame on which the failure avoidance process is performed can be reduced.
  • the necessary calculation of the subsequent frame to be encoded after the one frame is characterized by increasing the amount. For example, by adding m times (m is a real number greater than or equal to 1) or a real number n greater than 0, the subsequent frame is added to the element used for calculating the necessary calculation amount or the necessary calculation amount of the subsequent frame. Increase the required amount of computation.
  • the required calculation amount Kp is likely to be smaller than the actually required calculation amount for the subsequent frame.
  • the necessary amount of computation for the subsequent frame satisfies the actual amount of computation in order to increase the necessary amount of computation for the subsequent frame. The possibility increases and the failure phenomenon in subsequent frames can be avoided.
  • the moving image encoding or decoding processing system of the present invention functions as a moving image encoding or decoding means for encoding or decoding a moving image composed of a plurality of consecutive frames in units of frames.
  • Necessary amount of computation required for encoding or decoding one frame Necessary amount of computation for calculating Kp, and within the time Te previously assigned to the encoding / decoding processing of the one frame.
  • a moving image encoding or decoding processing system that performs encoding or decoding processing of the one frame by the ⁇ or decoding means
  • the second failure avoiding means is provided.
  • the moving image encoding or decoding processing method of the present invention encodes or decodes a moving image composed of a plurality of consecutive frame forces using a processor in units of frames.
  • Step a necessary computation amount calculating step for calculating a necessary computation amount Kp required for encoding or decoding one frame, and a time Te previously allocated to the encoding or decoding processing of the one frame.
  • An operation determining step for determining an operating frequency capable of encoding or decoding the necessary computation amount Kp, and the processor determines the operation determining step within the time period Te /! While operating with the obtained operating frequency and the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency, the video coding key or decoding step
  • the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount, the required calculation amount of the subsequent frame that is encoded or decoded after the one frame is increased.
  • the second failure avoidance step is provided.
  • the required calculation amount Kp is smaller than the actually required calculation amount.
  • the necessary calculation amount Kp is also smaller than the actually required calculation amount for the subsequent frames.
  • the second failure avoiding means and the second failure avoiding step increase the necessary amount of computation for the subsequent frame, the necessary amount of computation for the subsequent frame may satisfy the actual amount of computation. It becomes higher and the failure phenomenon in the subsequent frame can be avoided.
  • the processing of the second failure avoiding means 'step is performed when the required calculation amount Kp of the predetermined frame is smaller than the actual calculation amount regardless of the presence or absence of the first failure avoiding means' step.
  • the second failure avoiding means and the second failure avoiding step may be performed by multiplying the necessary calculation amount of the subsequent frame or an element used for calculating the required calculation amount by m times (m is 1 or more). Real number) or a real number n greater than 0 is added.
  • the value of the required calculation amount calculated by the required calculation amount calculation means in the processing of the subsequent frame or the value of the element used for calculating the required calculation amount is m times (m is By adding a real number n greater than 1) or a real number n greater than 0, it is possible to increase the necessary amount of computation Kp of the subsequent frame and avoid the occurrence of a failure phenomenon in the subsequent frame.
  • m is By adding a real number n greater than 1) or a real number n greater than 1, it is possible to increase the necessary amount of computation Kp of the subsequent frame and avoid the occurrence of a failure phenomenon in the subsequent frame.
  • elements to be increased, and values of m and ⁇ it is preferable to derive appropriate elements from the experiment beforehand and set them in the system.
  • the moving image encoding or decoding processing system of the present invention functions as moving image encoding or decoding means for encoding or decoding a moving image composed of a plurality of consecutive frames in units of frames.
  • Necessary amount of computation required for encoding or decoding one frame Necessary amount of computation for calculating Kp, and within the time Te previously assigned to the encoding / decoding processing of the one frame.
  • the moving image encoding code is operated while operating at the operating frequency determined by the operation determining means and the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency.
  • it is a moving image encoding or decoding processing system that performs encoding or decoding processing of the one frame by decoding decoding means, and includes third failure avoiding means that extends the time Te at a predetermined timing. It is characterized by having.
  • the moving image encoding or decoding processing method of the present invention includes a moving image encoding step of encoding or decoding a moving image composed of a plurality of continuous frame forces using a processor in units of frames.
  • the necessary calculation amount calculation step for calculating the necessary calculation amount KP required for encoding or decoding one frame, and the time Te allocated in advance for the encoding or decoding processing of the one frame.
  • An operation determining step for determining an operating frequency capable of encoding or decoding the necessary calculation amount Kp, and
  • a third failure avoidance step is provided which extends the time Te at a predetermined timing.
  • the present invention when the encoding / decoding processing of one frame cannot be completed at the time Te allocated in advance, the time allocated to the processing of the next subsequent frame is used.
  • the processing time Te of one frame is extended, and the encoding / decoding processing of the one frame is completed. As a result, it is possible to avoid a failure phenomenon that does not deteriorate the image quality without reducing the actual processing amount of encoding or decoding.
  • the third failure avoiding means when extending the time Te previously assigned to the encoding process of the one frame, is adapted to return the code after the one frame. For the subsequent frame to be signaled, the time Te that is assigned in advance to the encoding process or the decoding process of the subsequent frame is changed.
  • the time allocated to the subsequent frame is shortened. !, The time obtained by subtracting the extended processing time assigned to the above one frame from the time assigned to the encoding or decoding processing of the subsequent frame, The operation of the processor that can process the coding or decoding of the subsequent frame within the time changed in accordance with the extension time to change the time Te, such as the time Te of decoding or decoding processing
  • the frequency is determined, and the processor operates at the operating frequency and the appropriate substrate bias voltage or operating power supply voltage. Therefore, low power consumption is realized in the encoding or decoding processing of subsequent frames. Even if the time Te assigned to the succeeding frame is shortened by extending the time Te assigned to one frame, changing the time Te of the succeeding frame may cause a failure in the processing of the succeeding frame. Can be lowered.
  • the third failure avoiding means stores an input frame for storing a frame to be encoded next when the time Te previously allocated to the encoding process of the one frame is extended. It is characterized in that the frame write destination is changed with respect to the memory.
  • the sequentially input frames are stored by changing the write destination without overwriting one frame. Therefore, the one frame can be held in the input frame memory, and the encoding process can be continued and completed.
  • the third failure avoiding means is provided for an input frame memory that stores a frame to be encoded next when the time Te previously allocated to the encoding process of the one frame is extended. The writing is impossible.
  • the input frame memory cannot write the frame, so that the subsequent frame is not overwritten.
  • the frame can be held, and the sign key process can be continued and completed.
  • the time Te Encoding processing can be performed on the same frame as when the power is not extended.
  • the moving image encoding system when performing a moving image encoding process, assuming that a frame to be encoded before the one frame is a previous frame.
  • the necessary calculation amount calculating means includes a motion amount between the predetermined frame and the previous frame, an amount of activity of the predetermined frame, an amount of activity of the previous frame, an average value of a quantization step size of the previous frame, and a quantization step of the previous frame.
  • the difference between the average value of the size and the average value of the quantization step size of the previous frame, the number of macroblock matching of the previous frame, the number of effective blocks of the previous frame, the number of effective coefficients of the previous frame, the sign of the previous frame The amount of computation actually required for the key, the number of bits generated in the previous frame, the sign key rate of the predetermined frame, and the predetermined frame.
  • the required calculation amount calculation means includes the number of bits of encoded data of a predetermined frame, the type of whether the predetermined frame is intra-frame encoded or inter-frame encoded, predetermined frame Or the average value of the motion vector size of the previous frame, the variance of the motion vector size of the predetermined frame or the previous frame, the number of effective blocks of the predetermined frame or the previous frame, the effective coefficient of the predetermined frame or the previous frame Number, the bit rate of the predetermined frame or the previous frame, the code amount of the predetermined frame or the previous frame, the average value of the quantization step size of the predetermined frame or the previous frame, the difference of the average value of the quantization step size (1 Difference in quantization step size of previous frame, or quantization step size of previous frame Difference between the quantization step size of the previous frame and the previous frame), the amount of calculation actually required for
  • Each of the plurality of elements is an element that affects the required amount of computation Kp in the encoding or decoding process.
  • the required amount of calculation Kp calculated by the required amount of calculation means is The value is closer to the amount of computation when encoding or decoding is actually performed. Therefore, it is less likely that the calculated required calculation amount Kp is too large compared to the actual calculation amount and the reduction in power consumption is hindered. Also, the required calculation amount Kp is smaller than the actual calculation amount and the sign It is difficult for the failure phenomenon that the defect or decryption process is not completed in time.
  • the amount of computation required for the encoding process of one frame is determined by the number of executions of element processing such as macroblock matching.
  • Element processing is classified into processing that is repeatedly executed for the number of times of macroblock matching, processing that is repeatedly executed for the number of effective blocks, and processing that is repeatedly executed for the number of effective coefficients.
  • the amount of computation required for one frame of code processing is the maximum (worst).
  • the number of macroblock matching, the number of effective blocks, and the number of effective coefficients fluctuate below the maximum value for each frame, so the amount of computation required for code processing must vary from frame to frame. It becomes.
  • the number of macroblock matching, the number of effective blocks, the number of effective coefficients, and the amount of computation required for encoding processing are close to each other between frames that are close in time.
  • the number of macroblock matching and the number of effective blocks of the previous frame that has already completed the encoding process at the time of prediction By adopting the number of effective coefficients and the amount of computation required for the encoding process, the required amount of computation K for each frame can be approximated to the amount of computation when the encoding process is actually performed. .
  • the amount of computation required for the decoding process for one frame is also determined by the number of executions of the element process of the decoding process.
  • Element processing is classified into processing that is repeatedly executed by the number of effective blocks, and processing that is repeatedly executed by the number of effective coefficients. When all of the number of effective blocks and the number of effective coefficients are maximized, one-frame decoding is performed. ⁇
  • the amount of computation required for processing is the maximum (worst). In the actual decoding process, the number of effective blocks and the number of effective coefficients fluctuate below the maximum value for each frame, so the amount of computation required for the decoding process varies for each frame.
  • the amount of computation required for the decoding process varies from frame to frame
  • the parameters used to predict the required computational complexity of the decoding process the number of effective blocks, the number of effective coefficients, and the number of effective coefficients of the previous frame that have already been decoded at the time of prediction
  • the required amount of computation K for each frame can be approximated to the amount of computation when decoding is actually performed.
  • the video encoding processing system For one frame to be encoded or decoded (frame to be encoded or decoded in the future), a calculation for predicting the necessary amount of computation Kp required for the code or decoding is performed, and the predetermined value is calculated.
  • the time Te allocated for frame processing is the minimum or close operating frequency required to process the required amount of computation K p, and the operating frequency and operating power supply voltage or operating frequency and substrate bias voltage for each frame. Or operating frequency
  • the substrate bias voltage and the operating power supply voltage are dynamically controlled, low power consumption can be realized.
  • the failure avoiding means since the failure avoiding means is provided, it is possible to avoid the failure phenomenon that occurs when the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount. Alternatively, it is possible to prevent the moving image subjected to the decoding process from being deteriorated. In addition, when processing by the failure avoidance means is executed, mitigation processing is performed by mitigation means that mitigates the effect on subsequent frame encoding processing, thereby suppressing image quality deterioration due to subsequent frame encoding processing. Can do.
  • FIG. 1 is a schematic block diagram showing the operation of the moving image code processing system according to the first embodiment of the present invention.
  • FIG. 2 is a diagram showing an implementation example of a moving image code key processing system according to the first embodiment of the present invention.
  • FIG. 3 is a diagram showing a schematic flowchart of a moving image encoding processing program that causes a computer to function as the moving image encoding process system of the embodiment.
  • FIG. 4 is a cross-sectional view showing a triple tool structure.
  • FIG. 5 is a conceptual diagram showing the operation power supply voltage / substrate bias voltage / operation frequency of the processor used in the moving image code processing system of the embodiment.
  • FIG. 6 is an explanatory diagram for explaining that power consumption can be reduced by keeping the operating power supply voltage and operating frequency constant.
  • FIG. 7 is an explanatory diagram for explaining the relationship between the time for performing an interrupt and the remaining calculation amount in the embodiment.
  • FIG. 8 is a schematic block diagram showing the operation of the moving image code processing system of the second embodiment of the present invention.
  • FIG. 9 is a diagram showing a schematic flowchart of a moving image encoding processing program that causes a computer to function as the moving image encoding process system of the embodiment.
  • FIG. 10 is a schematic block diagram showing an operation of a video decoding process system according to a third embodiment of the present invention.
  • FIG. 11 causes a computer to function as the moving image code processing system of the above embodiment.
  • FIG. 12 is a graph showing a processing time transition of normal encoding processing.
  • FIG. 13 is a diagram showing a transition of processing time of sign key processing in the third embodiment.
  • FIG. 14 is a diagram showing a processing time transition of the sign key processing in the third embodiment.
  • FIG. 15 is a diagram showing a transition of processing time of sign key processing in the third embodiment.
  • FIG. 16 is a schematic block diagram showing the operation of the moving image decoding system according to the fourth embodiment of the present invention.
  • FIG. 17 is a schematic block diagram showing the operation of the moving picture code processing system according to the fifth embodiment of the present invention.
  • FIG. 18 is a conceptual diagram illustrating the relationship between the operation power supply voltage and the operation frequency of the processor used in the moving image code processing system of the above embodiment.
  • FIG. 19 is a schematic block diagram showing an operation of a moving image code processing system according to a sixth embodiment of the present invention.
  • FIG. 20 is a conceptual diagram for explaining a relationship between a substrate bias voltage and an operating frequency of a processor used in the moving image code processing system of the embodiment.
  • FIG. 21 is a diagram showing an example of the relationship between the operating frequency of the processor, the operating power supply voltage, and the substrate bias voltage in the example.
  • FIG. 22 is a diagram showing a conventional technique for reducing power consumption in a moving image code processing system.
  • FIG. 23 is a conceptual diagram showing a state in which the amount of calculation of a moving image code key or decoding key varies from frame to frame.
  • Video coding means 4, 54, 64 Operation control means 5 Video coding means
  • the moving image encoding / decoding system of the present invention is such that a processor 1 described later performs moving image encoding processing and moving image decoding processing.
  • moving image encoding processing is performed. It functions as a system and functions as a video decoding processing system when performing video decoding.
  • the moving image encoding / decoding processing system of the present invention may be one that performs encoding or decoding in units of frames or in units of time, or only decoding processing or only encoding processing. It may be something that performs.
  • processor 1 is a computing unit dedicated to moving image processing (command) for efficiently performing moving image coding processing or moving image decoding processing with a smaller number of cycles, smaller power consumption, and smaller program code amount.
  • arithmetic unit (instruction) dedicated to moving image processing a product-sum arithmetic unit (product-sum operation instruction) used in matrix operations such as discrete cosine transform processing, motion vectors
  • the difference absolute value sum operator (difference absolute value sum command) used in operations such as block matching operations in the detection process is given below.
  • the case where decoding is performed is a moving image decoding system, which will be described in detail separately for moving image encoding processing and moving image decoding processing.
  • the moving image coding processing system S1 has a predetermined frame that occurs when the required calculation amount Kp calculated by the required calculation amount calculation means 2 is smaller than the actual calculation amount of the predetermined frame.
  • the code of the predetermined frame is allocated in advance to the encoding process of the predetermined frame.
  • the first failure avoiding means 9 is provided to perform processing to avoid the failure phenomenon, and the operating frequency, the base bias voltage, and the operating power supply voltage are controlled to be constant for each frame.
  • This system S1 is realized by a computer which is an information terminal device such as a mobile phone or a personal computer with a built-in microcomputer, for example, and is particularly a system that functions as a part of a multimedia signal processing unit in the computer. There is a system that sequentially encodes a moving image composed of a predetermined number of frame forces in units of frames.
  • Fig. 1 is a schematic block diagram showing the operation of the video code key processing system S1 of the present embodiment
  • Fig. 3 shows a video code key processing method realized by the system S1.
  • the moving image coding processing system S1 is variable in operating power supply voltage Vdd and substrate bias voltage Vbn, Vb P and operating frequency F force S r (r is an integer of 2 or more) (ie, r operating power supply) Voltage Vdd and substrate bias voltage Vbn, Vbp, and operating frequency F), and the operating power supply voltage and substrate bias voltage and operating frequency can be changed by the program, and the operating power supply voltage and substrate bias of processor 1
  • a computer in particular, a computer having at least an operation control means 4 for controlling a voltage and an operating frequency, and a local decoding frame memory 6, an input frame memory 7, and an element memory 8 which are storage areas for storing predetermined data.
  • Vbn is the substrate bias voltage of the n-channel MOS transistor and Vbp is the p-channel MOS transistor.
  • Substrate bias voltage In the local decoding memory 6 and the input frame memory 7 and the like, the operation voltage “operation frequency” and the substrate bias voltage may be controlled by the operation control means 4 in the same manner as the processor 1.
  • the elements included in control area CA indicated by dotted lines The operating frequency, operating voltage, and substrate bias voltage are controlled.
  • the processor 1 is a semiconductor element having a triple-well structure, and the substrate bias voltage can be controlled for both the nMOS transistor and the pMOS transistor.
  • the local decoding memory 6 and the input frame memory 7 are semiconductor memory elements, and the operation control means 4 controls the operation power supply voltage / substrate bias voltage / operation frequency in the same manner as the processor 1.
  • FIG. 4 is a partial cross-sectional view of the processor 1 having a triple-well structure.
  • the processor 1 has a triple-well structure by forming an n-type well n-well on a p-type semiconductor substrate p-sub and further forming a p-type well p-well on an n-type well n-well. is there.
  • an n-channel MOS transistor and a p-type well contact layer p-Contact are formed in the p-type well p-well.
  • the n-channel MOS transistor has a source Z drain layer S, D composed of an n-type impurity layer, and a gate electrode G.
  • n-type n-well In the n-type n-well, a p-channel MOS transistor and an n-type contact layer n-contact are formed.
  • the n-channel MOS transistor has a source Z drain layer S, D having a p-type impurity layer force, and a gate electrode G.
  • a substrate bias voltage V bn is applied to a p-type well p-well, which is a semiconductor region where an n-channel MOS transistor is formed, via a p-type contact contact layer p-Contact.
  • a substrate bias voltage Vb p is applied to an n- type well n — well, which is a semiconductor region where a p-channel MOS transistor is formed, via an n- type well contact layer n — Contact.
  • the operation control means 4 includes an operation power supply voltage control means 4c having a DC-DC converter, etc., a substrate bias voltage Vbn generation means 4d for controlling the substrate bias voltage of the n-channel MOS transistor, p-channel Substrate bias voltage Vbp generation means 4e for controlling the substrate bias voltage of MOS transistors 4e, operating frequency control means 4b with PLL, etc.
  • a substrate bias voltage Vbn generation means 4d for controlling the substrate bias voltage of the n-channel MOS transistor
  • p-channel Substrate bias voltage Vbp generation means 4e for controlling the substrate bias voltage of MOS transistors 4e
  • operating frequency control means 4b with PLL etc.
  • each element of the operation control means 4 may exist outside the moving image encoding processing system SI, and may control the operating power supply voltage, the substrate bias voltage, or the operating frequency from outside the moving image encoding processing system S1.
  • the processor 1, the memories 6 and 7, and the operation control means 4 are connected to each other via wiring.
  • the processor 1 includes necessary calculation amount calculation means 2, operation determination means 3, moving picture encoding means 5, and first failure avoidance means 9 as means that operate on the processor 1.
  • Reference numeral 101 is input image data
  • reference numeral 102 is an operation power supply voltage and substrate bias voltage and operation frequency instruction
  • reference numeral 103 is local decoded data of the previous frame
  • reference numeral 105 is operation power supply voltage 'base bias voltage' operation frequency Supply
  • code 106 is the encoded data of the frame
  • code 107 is information on the average value of the quantization step size of the previous frame
  • code 108 is a force inter-frame code that is an intra-frame code for each frame.
  • the code 109 is the information on the video code bit rate
  • the code 110 is the activity amount of the previous frame
  • the code 111 is the number of macroblock matching in the previous frame
  • the code 112 is the number of effective blocks in the previous frame
  • the code 113 Is the number of effective coefficients of the previous frame
  • reference numeral 114 is the average of the quantization step size of the previous frame and the average of the quantization step size of the previous frame. Difference between the values, symbols 115 are code processing amount actually required to I spoon, reference numeral 116 necessary calculation amount of the previous frame calculated by necessary calculation amount calculations means 2 of the previous frame.
  • the element memory 8 is a part of a plurality of elements used in the required amount-of-computation calculation means 2 described later (intra-frame code ⁇ type 108 or code ⁇ . This is a storage area for storing the bit rate 109, the amount of frame activity 110, and the required amount of calculation 116) calculated by the required amount of calculation calculation means 2.
  • the processed macroblock number register 10 is a register that temporarily stores information on the number of macroblocks 117 that have been subjected to sign processing.
  • MPEG-4 is used as the video encoding method for moving image encoding means 5, but other encoding schemes such as H.26X, MPEG-1 and MPEG 2 can be used! ,.
  • FIG. 2 shows an implementation example of the video code key processing system S1.
  • System S1 mainly consists of processor 1, various memories MR, 7a, 7b as peripheral devices, various interfaces CI, DI, BI, operation control circuit 4a, PLL4b, DC-DC converter 4c, substrate bias voltage generation circuit Realized by hardware equipped with 4d, 4e, etc.
  • Each of the above components includes buses Bl, B2, etc. It becomes possible to communicate with each other via!
  • the processor 1 includes a processor core la, an instruction cache memory lb, a data cache memory lc, and a bus controller BC.
  • Necessary calculation amount calculation means 2, motion determination means 3, video encoding means 5, failure avoidance means 9, and 11 are executed by executing a program stored in the memory MR as necessary on the processor core 1a. Realized.
  • the instruction cache memory lb and the data cache memory lc are cache memories provided for high-speed processing of programs executed on the processor core la.
  • the local decoding frame memory 6, the element memory 8, and the processed macroblock number register 10 are aggregated in the memory MR of Fig. 2 and the average value of the quantization step size of the previous frame 107, for each frame.
  • the actual amount of processing required for the sign of the previous frame 115, the required amount of previous frame 116 calculated by the required amount calculation means, and the number of processed macro blocks 117 are stored in the memory MR. It is stored in.
  • Locally decoded data 103 is transmitted and received as signals 100j, 100k, and 1001 between memory MR and processor core la via bus controller BC.
  • the two input frame memories 7a and 7b correspond to the input frame memory 7 of FIG.
  • Video data (input image data 101) input from the camera interface CI is input to the input frame memory 7a (or input frame memory 7b) via the bus B2.
  • the input frame memory (# 0) 7a and the input frame memory (# 1) 7b are switched in use every time one frame is processed. In other words, in the processing of the i-th frame, the input image data is written to the input frame memory (# 1) 7b by the signal 100h, and the input frame memory is input by the signal 100 ⁇ for the encoding process by the moving image encoding processing means.
  • the input image data is read from (# 0) 7a
  • the input image data is written to the input frame memory (# 0) 7a by the signal 100i in the processing of the (i + 1) -th frame, and the moving image is encoded.
  • input image data is input from the input frame memory (# 1) 7b by the signal 100p. Data is read. Therefore, the signal ⁇ does not occur when the input image data is written to the input frame memory (# 1) 7b by the signal lOOh, and conversely, the signal 100h does not occur when the image is read by the signal ⁇ . .
  • the input frame memory (# 0) 7a is used for the processing of the i-th frame
  • the input frame memory (# 1) 7b is used for the processing of the (i + 1) -th frame.
  • Voltage and operating voltage are subject to control.
  • the input frame memory is prepared for two frames, and each operation frequency can be set independently, so that the input image data is always written from the camera interface CI at a constant operation frequency. As a result, it is possible to execute the reading operation of the input image data whose operating frequency varies based on the calculated value of the necessary calculation amount without interfering with each other.
  • the operation control circuit 4a can transmit and receive signals to and from the PLL 4b, the DC-DC converter 4c, and the substrate bias voltage generation circuits 4d and 4e, and these function as the operation control means 4.
  • the operation control circuit 4a receives the operation power supply voltage 'substrate bias voltage' operation frequency instruction 102 by the signal lOOe from the processor core la, and transmits the signal lOOu to the PL L4b based on the instruction 102, and the DC-DC converter Signal ⁇ is transmitted to 4c, signal lOOw is transmitted to substrate bias voltage generation circuit 4d, and signal ⁇ is transmitted to substrate bias voltage generation circuit 4e.
  • the PLL 4b transmits the operating frequency signal 100a based on the signal lOOu
  • the DC-DC converter 4c supplies the operating power voltage 100b based on the signal ⁇
  • the substrate bias voltage generating circuit 4d generates nM based on the signal lOOw.
  • the OS substrate bias voltage 100c is supplied, and the substrate bias voltage generation circuit 4e supplies the pMOS substrate bias voltage 100d based on the signal ⁇ .
  • Signals 100e, lOOj, 100k, 1001, 100m, 100 ⁇ , lOOp, 10 Oq, lOOr, 100s are the operating frequency signals output by PLL4b 100a, DC—DC comparator Power supply voltage output 100b output from the substrate 4c, nM OS substrate bias voltage 100c output from the substrate bias voltage generation circuit 4d, pMOS substrate output from the substrate bias voltage generation circuit 4e
  • the frequency and signal level depend on the value of the bias voltage 100d Change.
  • Code key data 10 6 after being encoded by the moving image code key means 5 operating on the processor 1 is transmitted as a signal 100m to the bit stream interface BI via the bus B1 and output as a signal 100 ⁇ .
  • the local decoded data 106 generated in the process of the code decoding process is transmitted as a signal 100j to the memory MR functioning as the local decoded frame memory 6.
  • image data and the like are read out from the memory as a signal 100q via the bus B1 and transmitted to the display interface DI.
  • the signal 100q received by the display interface DI is output as video data by the signal 100t.
  • Video data is output and displayed as moving images via a monitor connected to the display interface DI.
  • the operation control circuit 4a, the display interface DI, and the bit stream interface BI always operate at a constant operation power supply voltage, but signals 100e, 100q, and 100m transmitted and received between them are elements included in the control area CA.
  • the signal level fluctuates according to changes in the substrate bias voltage and operating power supply voltage (processor 1, memory MR, input frame memory 7a, 7b, etc.).
  • the operation control circuit 4a, the display interface DI, and the bit stream interface BI include a level converter that corrects the signal level of the received signals 100e, 100q, and 100m.
  • the moving image code processing system S1 is realized by causing a computer (in particular, a multimedia signal processing unit in the computer) to function as the following predetermined means by the moving image code processing program Prgl. Further, the system S1 realizes the moving picture coding method of the present invention having the following step force.
  • a predetermined frame that is, a frame to be encoded next when a certain frame is encoded.
  • the code is not processed yet, and the frame that is scheduled to be processed in the future (also called the current frame) is encoded before the predetermined frame.
  • Taichi no frame past The process of encoding a predetermined frame with the previous frame as a previous frame
  • the same process is performed for V and shifted frames.
  • the moving image encoding processing program Prgl causes the computer to function as follows in steps 1 to 5 described later.
  • Step 1 Input image information of a predetermined frame into the input frame memory 7.
  • Step 2 The required calculation amount Kp for a predetermined frame is calculated.
  • Step 3 It is made to function as the operation determining means 3 for determining the operating frequency F, the operating power supply voltage Vdd, and the substrate bias voltage Vb n, Vbp of the processor according to the calculated necessary calculation amount ⁇ .
  • Step 4 It is made to function as the operation control means 4 for controlling the operation of the processor 1 with the calculated operation frequency F, operation power supply voltage Vdd, and substrate bias voltage Vbn, Vbp.
  • Step 5 It is made to function as the moving image encoding means 5 for encoding the image information of a predetermined frame. As described above, the processing of step 1 and step 5 is performed for all frames in the order of frames input to the input frame memory 7 (that is, the order of encoding), thereby encoding the moving image. Details will be described below.
  • Step 1 The inputted input image data is stored in the input frame memory 7 which is a storage area for temporarily storing the frames in order to synchronize the frames.
  • Necessary computation calculation means 2 accesses the input frame memory 7 to acquire the input image data 101 of the predetermined frame, and is necessary for the encoding processing of the predetermined frame. Calculate the necessary amount of computation Kp.
  • There are various methods for calculating the required amount of computation ⁇ For example, it is desirable to calculate by using one or more elements that affect the amount of computation of the sign ⁇ processing of a predetermined frame. As an element, for example, in the case of moving image code processing, when the processing content is motion compensation, the amount of computation is large for a video with a lot of motion, while the amount of computation is small for a video with little motion.
  • the distortion value calculated by the sum of absolute differences as the amount of motion between the predetermined frame and the previous frame the value calculated by the sum of absolute differences of adjacent pixels as the amount of activity of each frame, and the macro
  • the number of effective blocks, the number of effective coefficients, the encoding bit rate, the number of generated bits, the calculation amount actually required for the sign of the previous frame, and the required calculation amount calculation means 2 The required amount of calculation of the previous frame calculated can be mentioned.
  • each element Assuming that only the value of one element changes and the value of the other element does not change for each, the required amount of computation is relatively smaller when the value of that one element is large than when it is small. The value of one element is small!
  • the required amount of computation is relatively small compared to the case.
  • the intra-frame code ⁇ is required when the required calculation amount Kp is relatively small compared to the case of an inter-frame code ⁇ .
  • the required amount of computation ⁇ is made relatively large. That is, since these multiple elements are elements that affect the required amount of calculation necessary for the sign processing of a predetermined frame, the required amount of calculation calculation means 2 determines the required amount of calculation ⁇ according to these elements. By calculating so as to increase or decrease (cycle), the required calculation amount ⁇ calculated by the required calculation amount calculation means 2 becomes a value closer to the calculation amount when the actual sign processing is performed.
  • calculation is performed using the function G, and the input image data 101 of a predetermined frame stored in the input frame memory 7 and the local decoding frame memory 6 are stored.
  • the motion magnitude of the input image is predicted (calculated).
  • the local decoded data 103 of the previous frame is encoded by the previous frame formed by encoding the previous frame in accordance with the code key processing of the previous frame in which the code is performed before the predetermined frame.
  • the data 106 is formed by decoding with a local decoder and stored in the local decoding frame memory 6. For example, the sum of absolute differences is used as an example of the prediction (calculation) of the magnitude of movement.
  • the local decoded data 106 decoded by the local decoder after encoding may be used, or the input image data of the input previous frame may be used as it is. good.
  • Input image data 101 of a predetermined frame stored in the input frame memory 7 is represented by X (i, j) (i is a horizontal coordinate of the image, j is a vertical coordinate), and a local decoded frame memory 6 to be described later.
  • the local decoding data 103 of the previous frame stored in is Y (i, j) (where i is the horizontal coordinate of the image and j is the vertical coordinate)
  • the amount of motion between the predetermined frame and the previous frame is the difference
  • Z ⁇ IX (i, j) -Y (i, j) I for all (or sampled) pixels. Let Z be the value of the sum of absolute differences.
  • the value of the adjacent pixel difference absolute value sum (that is, the activity amount of each frame) is set to w.
  • the sum of absolute differences is Z
  • the amount of activity of the predetermined frame is Wa
  • the amount of activity of the previous frame past frame
  • Wb the amount of activity of the previous frame
  • Wb the amount of activity of the previous frame
  • the average quantization step size of the previous frame (average value of quantization step size) Qprev
  • M for the number of macroblock matching in the previous frame
  • B for the number of effective blocks for the previous frame
  • C for the number of effective coefficients for the previous frame
  • S for the amount of processing actually required for the sign of the previous frame
  • BR is the frame code bit rate
  • a Qprev is the difference between the average quantization step size of the previous frame and the average quantization step size of the previous frame, and the actual number of bits generated in the previous frame.
  • Kp G (Z, Wa, Wb, Qprev, M, B, C, S, BR, ⁇ Qprev, D, Kp,)
  • G is Z, Wa, Wb, Qprev, M, B, C, S, BR, ⁇ Qprev, D,
  • a function derived from one or more elements of Kp ' is derived from one or more elements of Kp '.
  • type I is used as to whether the predetermined frame is a force inter-frame code ⁇ that is an intra-frame code ⁇ .
  • the function G will be described.
  • the image change between the previous frame and the predetermined frame is large (small), that is, when the sum of absolute differences Z is large (small)
  • the number of macroblock matching performed in the predetermined frame becomes large (small).
  • the amount of computation required for motion detection processing for a given frame (depending on the number of macroblock matching operations to be performed Becomes larger (smaller).
  • the activity amount Wa of the predetermined frame is large (small), it means that the predetermined frame includes a lot (small) of high frequency components of the image.
  • the function G is configured so that Kp is set large (small) when parameters such as Z and Wa are large (small).
  • the above function G is configured to set Kp large (small) when parameters such as M, B, C, S, Wb, Kp 'are large (small).
  • the value of the quantization step size is set small (large).
  • the number of effective blocks and the number of effective coefficients generated in the encoding process are large ( Small).
  • the quantization step size value for a given frame is set small (large), which is an effective value generated in the encoding process.
  • the number of blocks and the number of effective coefficients become smaller (larger).
  • the above-mentioned function G has a larger number of generated bits D in the previous frame compared to BR so that Kp is set to be large (small) when the code key bit rate BR of the predetermined frame is large (small). If (small ⁇ ), configure Kp to be small (large). Furthermore, the difference between the average quantization step size Qprev of the previous frame and the average quantization step size of the previous frame and the average quantization step size of the previous frame. By considering A Qprev, Kp calculated by the above function G can be made close to the amount of computation required to actually encode a predetermined frame.
  • the required amount of calculation 116 of the previous frame calculated by the calculation amount calculation means is stored in advance in the element memory 8, which is a storage area for storing elements, and is read into the calculation amount calculation means 2 when calculating the required calculation amount Kp. Used.
  • Average value of quantization step size of previous frame 107, number of macroblock matchings of previous frame 111, number of effective blocks of previous frame 112, number of effective coefficients of previous frame 113, average value of quantization step size of previous frame The difference between the average value of the quantization step size of the previous frame and the average value 114 of the previous frame and the amount of processing 115 actually required for the previous frame code It is fed back from the image code key means 5 to the necessary calculation amount calculation means 2.
  • the necessary calculation amount calculation means 2 only one of these elements may be used, or a plurality of elements may be used in combination.
  • Step 3 Operation Determination Step
  • the operation determination means 3 performs calculation for predicting the operation frequency Fe (cycle Z seconds) for the processing of a predetermined frame based on the value of the necessary calculation amount Kp.
  • the time Te allocated to the code processing for a given frame is the time Tp for predicting the amount of computation for a given frame from the time limit Tf for processing one frame and the operating frequency of the processor 'operating power supply voltage ⁇ board
  • the time to change the bias voltage is the time minus Ts.
  • the operating power supply voltage, substrate bias voltage, and operating frequency supported by peripheral devices including processor 1 and / or local decoding memory 6 are changed in r stages (r is an integer of 2 or more).
  • the operation determining means 3 performs an operation in which F (n)> Fe and Fe> F (n— 1).
  • Calculation is performed to select the frequency F (n) as the operating frequency for encoding the predetermined frame, and the operating power supply voltage Vdd (n) and the substrate bias voltage Vbn, Vbp (n) suitable for the operating frequency F (n) are calculated. Perform calculations to select and operate peripheral devices including processor 1 and / or local decoding memory 6 etc. at operating frequency F (n), operating power supply voltage Vdd (n), and substrate bias voltage Vbn, Vbp (n) The operation control means 4 is instructed to supply the operation power supply voltage 'substrate bias voltage' to the operation frequency (reference numeral 102). N is an integer between 1 and r
  • FIG. 5 is a diagram showing the relationship between the operating frequency “operating power supply voltage” and the substrate bias voltage.
  • the operation power supply voltage 'substrate is set so that the current consumed by the peripheral device including the processor 1 or the processor 1 and the local decoding memory 6 is equal to or less than a predetermined value for each operating frequency.
  • a combination of bias voltages is preset. For example, based on the relationship between the subthreshold leakage current 1st and the charge / discharge current led, the operating power supply voltage Vdd and the substrate bias voltage Vbn, Vbp that minimize the power consumption P for each operating frequency are obtained by experiments and calculations. A combination of power supply voltage Vdd and substrate bias voltage Vbn, Vbp is desirable.
  • the substrate bias voltage may be automatically calculated with respect to the operating power supply voltage corresponding to the operating frequency by hardware and / or a program built in the operation determining means 3. Further, the operating power supply voltage and the substrate bias voltage may be calculated with respect to the operating frequency by hardware and / or a program built in the operation determining means 3.
  • the operation control means 4 receives the operation power supply voltage V dd (n), the substrate bias voltages Vbn (n) and Vbp (n) and the operation frequency F (n) received from the operation determination means 3 Is supplied to peripheral devices including processor 1 and / or local decoding memory 6 (symbol 105), and its operating power supply voltage Vdd (n) and substrate bias voltage Vbn (n), Vbp (n)
  • the processor 1 and the peripheral device are controlled to operate at a constant frequency F (n).
  • the peripheral device including the processor 1 and / or the local decoding memory 6 and the like has a constant operating power supply voltage Vdd (n), a substrate bias voltage Vbn (n), Vbp (n), and an operating frequency F (n). Will work.
  • the operation power supply voltage control means 4c built in the operation control means 4 is constant.
  • the processor 1 is controlled to operate at the operating power supply voltage Vdd (n), and the processor 1 is controlled to operate at the constant substrate bias voltage Vbn (n) for the n-channel MOS transistor by the substrate bias voltage Vbn control means 4d. ⁇
  • the substrate bias voltage Vbp control means 4b controls the processor 1 to operate at a constant substrate bias voltage Vbp (n) for the p-channel MOS transistor, and the operating frequency control means (PLL) 4b controls the constant operating frequency F.
  • PLL operating frequency control means
  • the processor 1 is controlled to operate constantly.
  • Substrate bias voltage control applies p-channel MOS transistor by applying appropriate substrate bias voltage Vbn (n) to n-channel MOS transistor for substrate bias voltages Vbn (n) and Vbp (n). Is applied by applying an appropriate substrate bias voltage Vbp (n).
  • the potential difference between the substrate bias voltage Vbn (n) for the n-channel MOS transistor and the ground potential Vss is Vbbn (n)
  • the substrate bias voltage Vbp (n) and the operating power supply voltage Vdd for the p-channel MOS transistor is Vbbp (n). That is,
  • Vbn (n) Vbbn (n) + Vss
  • Vbp (n) Vbbp (n) + Vdd (n)
  • Vbbn (n) and Vbbp (n) and the operating power supply voltage Vdd (n) can be set independently.
  • Vbbn (n) is the voltage applied to the drain-source pn junction of the n-channel MOS transistor, and this voltage must not exceed the diffusion potential ⁇ ⁇
  • Vbb p (n) is the p-channel This is the voltage applied to the pn junction between the drain and source of the transistor, so that this voltage does not fall below the diffusion potential V ⁇ .
  • the diffusion potential V ⁇ is usually 0.6 V.
  • Step 5 moving picture coding step
  • the moving picture coding means 5 is a means realized on the processor 1 of the computer by the moving picture coding processing program Prgl. This is a means for accessing the input image data stored in the frame memory 7 in units of moving image encoding and performing the encoding process. That is, the moving image encoding means 5 acquires the input image data 101 of a predetermined frame from the input frame memory 7 and encodes it to generate the encoded data 106.
  • the peripheral devices including the processor 1 and / or the local decoding memory 6 and the like operate at a constant operating voltage supplied from the operation control means 4.
  • the operation control means 4 While operating peripheral devices including the processor 1 and / or the local decoding memory 6 at the operating frequency F (n), the operating power supply voltage Vdd (n), and the substrate bias voltage Vbn (n), Vbp (n) Therefore, the moving image code key means 5 that performs the code key using the processor 1 performs the key key code.
  • peripheral devices including the processor 1 and / or the local decoding memory 6 etc. are constantly operated at a high frequency for an image (input image data 101 of a predetermined frame).
  • the moving picture encoding unit 5 includes a local decoder having a function of decoding the code key data 106.
  • the code key data 106 of a predetermined frame is decoded by the local decoder and is then decoded in the local decoding frame memory 6 Is stored as locally decoded data 103.
  • the local decoded data 103 of the predetermined frame is used when calculating the necessary calculation amount Kp for the frame encoded next to the predetermined frame.
  • the encoded data 106 of a predetermined frame is transmitted through a transmission path or stored in a storage medium.
  • the operation power supply voltage and the substrate bias voltage are controlled according to the operation frequency F by the operation power supply voltage Vdd, the substrate bias voltage Vbp of the p-channel MOS transistor, and the n-channel MOS transistor substrate bias voltage Vbn. Only at least one voltage may be controlled.
  • the operation frequency 'operating power supply voltage' used for the processing of one frame, and the operation determining means 3 that only requires the substrate bias voltage to operate at a constant level is set to N sets of operating frequency 'operating power supply voltage' substrate bias voltage ( N may be a positive integer).
  • N may be a positive integer.
  • the operating time of each of N sets of operating frequencies “operating power supply voltage” and substrate bias voltage is calculated, and processor 1 is operated.
  • the code processing system S1 includes first failure avoiding means for avoiding the failure phenomenon, and the first failure avoiding step is executed by interrupting the moving image encoding step.
  • the first bankruptcy avoiding means is within the time Te when the code processing of a predetermined frame is allocated.
  • the process is switched to a process capable of reducing the amount of calculation of the normal code process so that the code process can be completed within the allocated time Te to avoid the failure phenomenon.
  • the timing Ti at which the process is switched is calculated by interrupting the sign process and temporarily interrupting the sign process.
  • the timing Ti may be updated according to the encoded processing.
  • Figure 7 shows the relationship between the time for interrupting and the remaining calculation.
  • Ks be the amount of computation required for processing.
  • the processor 1 reads from the processed macroblock number register 10 the number of macroblocks MBi (reference numeral 117) for which the sign process has been completed, and calculates the switching timing Ti using the following equation.
  • Example of First Failure Avoiding Means 1 The first failure avoiding means 91 of Example 1 is the same as the first failure avoiding means 91 in step 5, in which the moving picture code key means 5 is a code key processing routine for the input image data 101 of a predetermined frame.
  • the sign ⁇ is not finished at the timing Ti determined above.
  • a process of forcibly making only the color difference block an invalid block is performed on the black block. Only the color difference block of the macro block is forcibly converted to an invalid block, and the luminance block is processed in the same manner as normal, thereby preventing the image quality from being deteriorated. The image quality can be improved rather than forcibly making the block invalid and avoiding the failure.
  • Example 1 of first failure avoiding means 2 is the timing Ti determined as described above when executing the code processing routine of a predetermined frame.
  • step (3) intra-frame code processing is forcibly performed on macroblocks that have not been completed. Compared with the case where inter-frame coding is performed, since the motion vector detection process which occupies the most processing amount is not performed, the actual calculation amount can be reduced. In addition, since the sign key processing is performed up to the last macro block, the number of corresponding macro blocks increases compared to Example 1, and the image quality can be further improved.
  • Example 1 of first failure avoiding means 3 performs the timing Ti determined as described above when executing the code processing routine of a predetermined frame.
  • the inter-frame encoding process is performed forcibly with the size of the motion vector set to 0 for macroblocks that have not been completed.
  • Example 3 since the interframe coding process is performed without performing the motion vector detection process, the amount of calculation reduction is small and the number of corresponding macroblocks increases. The effect on the coding process of the next frame is Compared to Example 2, the smaller image quality can be further improved.
  • Example 4 of first failure avoiding means The first failure avoiding means 94 of Example 4 compulsorily applies to a macroblock whose code is not finished at the timing Ti determined above.
  • the quantization step size is increased and the sign key processing is performed.
  • the occurrence of the number of effective blocks and the number of effective coefficients for the macroblock is reduced, and among the encoding processes, the processes performed on the effective blocks and the variable length encoding performed on the effective coefficients. The number of processes can be reduced.
  • Each example of the first failure avoidance means described above omits the contents of the normal encoding process, forces the motion vector to 0, and suppresses the number of effective blocks and generation of effective coefficients. The amount is reduced. For this reason, the failure avoidance process by the first failure avoiding means in a predetermined frame is performed. If the interframe coding processing is performed on the macroblock of the next frame corresponding to the macroblock that has been subjected to the failure avoidance processing on the predetermined frame in the processing of the next frame, the image quality is affected. there is a possibility. For this reason, there is a possibility that the image quality of the next frame is deteriorated even though the image quality deterioration is prevented by the failure avoidance process at the predetermined frame.
  • the quantization is performed on the macroblock.
  • Relaxation processing is performed by the relaxation means 95 that performs the sign key processing by reducing the step size. This process increases the amount of generated code assigned to the corresponding macroblock, and increases the information obtained from the macroblock power to improve image quality.
  • the present invention can further reduce the power consumption due to the subthreshold leakage current as compared with the prior art in which one frame is encoded while changing the operating frequency of the processor a plurality of times.
  • the substrate bias voltage and operating frequency of processor 1 are variable in r steps as shown in Fig. 5, the required amount of computation for any one frame is Kt, and the time allotted for processing of that frame is Tt. .
  • the substrate bias voltages are Vbp and Vbn, and the thresholds are suitable for the substrate bias voltages Vbp and Vbn.
  • the substrate bias voltage is Vbp2 and Vbn2
  • the threshold voltage suitable for the substrate bias voltage Vbp2 and Vbn2 is Vt2
  • the required amount of computation at time T1 + T2 Case 2 is when the Kt processing ends.
  • Case2 consider the case of spoon codes I said arbitrary one frame. However, the threshold voltage is Vtl>Vt> Vt2, and it depends on the subthreshold leakage current. Power consumption is
  • Vdd Operating power supply voltage
  • Vgs Gate-source voltage
  • Vt Shiki! / Variable voltage
  • S Subthreshold swing
  • Pstl 10 " 2 : (10 _3 / 3 + 10" 1 X 2/3)
  • the present invention can further reduce the power consumption as compared with the prior art in which one frame is encoded while changing the operating power supply voltage and operating frequency of the processor a plurality of times. For example, when performing a certain amount of computation Kt at a certain time Tt Control at the same frequency during that particular time and the frequency Ft
  • the operating power supply voltage and operating frequency of processor 1 are variable in r steps as shown in Fig. 5, the required amount of computation for any one frame is Kt, and the time allotted for processing of that frame is Tt. .
  • the operating frequency is set to Ft
  • the operating power supply voltage when operating processor 1 at the operating frequency Ft is Vdd
  • the processing of the required amount of computation Kt is completed at time Tt. (I.e., when the operating frequency is constant) is set to Casel
  • the initial operating frequency is set to h * Ft, and the processor is operated at the operating frequency h * Ft.
  • the operating power supply voltage is VI
  • the processor operating frequency is changed to h * FtZ2 when time T1 has elapsed
  • the operating power supply voltage when operating processor 1 at the operating frequency h * FtZ2 is V2
  • the time Tl + Case 2 where the processing of the required amount of computation Kt ends at T2 (that is, when the operating frequency is switched once) is Case 2
  • power consumption is
  • an operating frequency that is greater than or equal to the predicted operating frequency and an operating frequency that is lower than the predicted operating frequency are selected, an interrupt is performed at a predetermined timing, and two steps are performed.
  • the operating frequency may be controlled.
  • the remaining amount of the required calculation amount of the predetermined frame calculated by the required calculation amount calculation means is actually used for the sign or decoding process of the predetermined frame by the code or processing unit. If it is smaller than the amount of computation required for the operation, the switching timing may be advanced so that the time for operating at a high V and operating frequency can be lengthened.
  • the moving image coding processing system S1 performs code coding processing after a predetermined frame when the calculation amount of code coding processing for a predetermined frame is reduced by the first failure avoidance means. It is also possible to increase the amount of calculation required for subsequent frames to be processed. This process of increasing the required amount of calculation for the subsequent frame is performed by the required calculation amount calculation means 2 for the subsequent frame in which the encoding process is performed after the predetermined frame. This is a process for increasing by a predetermined value, and is executed in the necessary calculation amount calculation step (Step 2) of the subsequent frame.
  • the final calculation is Necessary calculation amount Kp is
  • the required amount of calculation is calculated within the required amount of calculation calculation step for the subsequent frame. Multiply the coefficient of any element used in the function G (Z) by m times (m is a real number greater than 1). For example, if the macroblock matching count M has become smaller due to the execution of failure avoidance processing, using the above example, the required amount of computation Kp calculated in the next frame is
  • Kp j + (a + ⁇ ) XM + j8B + yC + ⁇ + ⁇ AQprev
  • Equation 1 (Equation 2), where all elements used in the function can be multiplied by m (m is a real number greater than or equal to 1), or real numbers n (n is a real number greater than or equal to 0) can be added to all elements. ), (Equation 3) may be increased by a fixed value regardless of the value of the required amount of computation.
  • FIG. 8 is a schematic block diagram of the video code key processing system S2 of the present embodiment
  • FIG. 9 is a schematic flowchart for explaining the video code key method realized by the system S2.
  • the moving image code processing system S2 of the present embodiment includes second failure avoiding means 96.
  • the second failure avoiding means 96 is actually required for the required calculation amount Kp calculated by the required calculation amount calculating means for the predetermined frame and the code amount processing for the predetermined frame after the sign key processing for the predetermined frame is completed. Compared with the calculated amount of computation, if “required amount of computation Kp ⁇ actual amount of computation required”, the subsequent frame in which encoding processing is performed after a predetermined frame is performed.
  • the video image code processing system S2 may include both the first failure avoiding means' step and the second failure avoiding means' step, or only the second failure avoiding means' step. Also good.
  • the second failure avoiding means 96 is a means that functions as a part of the required amount-of-computation calculation means 2, and is executed in the required amount-of-computation calculation step (Step 2). Specifically, the second failure avoiding means 96 compares the required calculation amount Kp calculated by the required calculation amount calculation means in the predetermined frame with the calculation amount actually required for the sign key processing of the predetermined frame. When “Required amount of computation Kp ⁇ actual amount of computation required”, the required computation amount of the succeeding frame is calculated according to the processing of the necessary computation amount calculating means for the subsequent frame in which the sign process is performed after the predetermined frame. Increase the amount Kp.
  • the required calculation amount Kp finally calculated is
  • the required amount of computation is calculated within the required computation amount calculation step for the subsequent frame. Multiply the coefficient of any element used in the function G (Z) by m times (m is a real number greater than 1). For example, macro block matching due to execution of bankruptcy avoidance processing If the number of times M has become small, using the above example, the required amount of computation Kp calculated in the next frame is
  • Kp j + (a + ⁇ ) XM + j8B + yC + ⁇ + ⁇ AQprev
  • Equation 1 (Equation 2), where all elements used in the function can be multiplied by m (m is a real number greater than or equal to 1), or real numbers n (n is a real number greater than or equal to 0) can be added to all elements. ), (Equation 3) may be increased by a fixed value regardless of the value of the required amount of computation.
  • the moving image code key processing system S3 replaces the first failure avoiding means with the sign code when the time allotted to the code key processing for a predetermined frame has elapsed.
  • a third failure avoiding means is provided for performing processing to avoid the failure phenomenon when the processing is not completed. The other points are the same as in the second embodiment.
  • FIG. 10 is a schematic block diagram of the video code processing system S3 including the third failure avoiding means 97
  • FIG. 11 is a schematic diagram for explaining the video encoding processing method realized by the system S3. It is a flowchart.
  • the third failure avoiding means 97 executes the encoding process when the allocated time Te elapses when the moving image code receiving means 5 executes the code processing routine of the input image data 101 of the predetermined frame in step 5. It is determined whether the ⁇ process has been completed or not, and if the sign process has not been completed, the time allocated to the next frame is used until the code process is completed. By extending the processing time of a predetermined frame, the encoding process cannot be completed! / And the failure phenomenon can be avoided.
  • the third failure avoiding means 97 will be specifically described below.
  • the input image data is stored in the input frame memory for each frame at the frame rate interval, and the input frame memory data force S is updated frame by frame at the input image frame rate interval.
  • the interval at which the data in the input frame memory is updated varies depending on the frame rate, and the time for processing the code for one frame is limited by the frame rate of the input image.
  • the input frame memory 7 is a memory that can store data for multiple frames (here, 2 frames), and the start address of the data is specified by the memory address, and the data is not specified. Address power shall be stored sequentially.
  • Fig. 12 is a diagram showing the transition of processing time in the case of normal encoding processing.
  • Fig. 13, Fig. 14 and Fig. 15 are diagrams of the code processing where the third failure avoiding means is executed by the third failure avoiding means. It is a figure which shows the processing time transition in a case. For example, in real-time processing, if 30 image data are input per second, the data in the external memory is updated every 1Z30 seconds. When encoding 10 images per second with the moving image encoding processing system, the input image is encoded at a rate of 1 out of 3 images.
  • the third failure avoiding means is for a case where the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount Km in the predetermined frame, that is, the time allocated to the sign key processing. In spite of the elapse of time, the encoding process is not completed.In contrast, the processing time Te of the predetermined frame is extended by using the encoding processing time assigned to the next frame! This is a means for controlling the status signal of the input frame memory in order to continue the encoding process and complete the process. Examples 3 and 2 below are examples of third means of avoiding bankruptcy.
  • the third failure avoiding means 97 has a function of executing Step 1 to Step 4 described later.
  • (Procedure 1) In order to extend the time Te allocated to the encoding process in a predetermined frame, the state of the input frame memory is controlled and the encoding process of the predetermined frame is continued.
  • (Procedure 2) When the encoding process for a given frame is completed, The time for which the frame sign can be processed is calculated.
  • (Procedure 3) Control switching of input frame memory status. Steps 1 to 3 are described in detail below.
  • FIG. 13 shows the transition of the sign key processing time by the failure avoiding means 97.
  • # 4 frame data is stored in the input frame memory 7b
  • # 7 frame data is stored in the input frame memory 7b.
  • the state of the input frame memory is controlled, the input frame memory 7b is controlled to be in a writable state, the input frame memory 7a is maintained in a writable state, and the frame data write destination of # 8 is set to the input frame. Change to memory 7a. Since the input frame memory 7b can hold the frame data of # 4 and the encoding process of # 4 can be continued, the time can be extended until the encoding process of # 4 is completed.
  • Step 2 Calculation of the next frame processing time
  • the required amount of calculation is calculated by the required calculation amount calculation means for the next frame to be encoded, and the operation frequency / operation power supply voltage and substrate bias voltage are calculated by the operation determination means.
  • the time T1 may be calculated when executing the necessary calculation amount calculation means for the next frame.
  • the sign key processing time T1 of the next frame it is possible to determine the optimum operating frequency for the sign key of the next frame, that is, the operating power supply voltage and the substrate bias voltage. The power consumption can be reduced efficiently. Note that the calculation of the time T1 may be performed when the necessary calculation amount calculating means for the encoding step of the next encoding target frame is executed.
  • Example of third failure avoiding means 2 The third failure avoiding means 98 is different from the third failure avoiding means 97 only in the state control method of the input frame memory. This is the same as the third bankruptcy avoidance measure 97.
  • FIG. 15 shows the time transition of the sign key processing by the third bankruptcy avoiding means 98.
  • the third failure avoiding means 98 when controlling the state of the input frame memory, if the processing cannot be completed in the encoding processing time Te assigned to # 4, at least the extension time To. During this time, the input frame memories 7a and 7b are controlled so that they cannot be written, and the time Te is extended by the time To.
  • the input frame memory holds the # 4 frame data stored in the input frame memory 7b and the # 7 frame data stored in the input frame memory 7a.
  • the failure phenomenon is avoided by extending the time Te.
  • the # 8 frame input during the extension time To is not written to the input frame memory.
  • the input frame memory 7b controlled to the writing state can write the input frame data.
  • the frame data # 7 is read out from the input frame memory 7a, and the frames # 9 and # 10 are sequentially written into the input frame memory 7b.
  • the encoding process of the # 7 and # 10 frame data is sequentially performed. Since the same frame number as that of the normal encoding process is encoded, an image quality equivalent to that of the normal encoding process can be obtained.
  • the encoded frames are # L # 2, # 3, # 4, # 5, # 6 , # 7, • ⁇ ', and the range in which the sign key processing time can be extended is the maximum Tf. # 4 encoding process If the extension time exceeds Tf, you may sign # 6 as the next frame.
  • the moving image decoding processing system S4 is a system for decoding an encoded moving image.
  • FIG. 16 is a schematic block diagram showing the operation of the video decoding processing system S4.
  • the operation power supply voltage, the substrate bias voltage, and the operation frequency are prepared in r stages (r is an integer of 2 or more), and the operation power supply voltage and the substrate bias are programmed.
  • the processor 1 capable of changing the voltage and the operating frequency, the operation control means 4 for controlling the operating power supply voltage, the substrate bias voltage and the operating frequency of the processor 1, and the local decoding frame memory 46 for storing the decoding data of the previous frame 46 With.
  • the operation power supply voltage, the substrate bias voltage, and the operation frequency may be controlled by the operation control means 4 in the same manner as the processor 1.
  • the processor 1 includes necessary calculation amount calculating means 42, operation determining means 3, moving picture decoding means 35, and third failure avoiding means 97 as means that operate on the processor 1.
  • Reference numeral 401 is input encoded data
  • reference numeral 102 is operating power supply voltage'substrate bias voltage / operating frequency instruction
  • reference numeral 105 is operating power supply voltage / substrate bias voltage / operating frequency supply
  • reference numeral 406 is decoding key data
  • the same reference numerals as those in the first embodiment are parts having the same function or equivalent functions.
  • the point of performing decoding key instead of code key is the same as that of the third embodiment except for the following points.
  • one of the frames that is sequentially decoded is arbitrarily decoded (that is, the frame that is decoded next on the basis of the time when a certain frame is decoded, in other words, At that time, a frame that has not been decoded yet and is scheduled to be decoded in the future) is a predetermined frame, and one frame that has been decoded before the predetermined frame (decoded in the past).
  • the moving picture decoding processing program Prg4 that causes the computer to function as the moving picture decoding processing system S4 is substantially the same as the moving picture code processing program Prgl.
  • Video decoding means 45 for decoding The computer (specifically, the processor 1 built in the computer) is made to function.
  • the input encoded data 401 input to the moving image decoding processing system S4 is input to the necessary calculation amount calculation means 42.
  • the required calculation amount calculation means 42 calculates the amount of information (number of bits) FB generated for one frame of the encoded data 401 (that is, the encoded data 401 of the predetermined frame), and predicts the required calculation amount Kp. (Required calculation step).
  • the required amount of computation ⁇ is
  • Kp G (FB, MVa, MVv, B, C, BR, Q, A Q, I, E, P)
  • FB is the number of bits of the code data of the predetermined frame or the previous frame
  • MVa is the average value of the motion vector of the predetermined frame or the previous frame
  • MVv is the size of the motion vector of the predetermined frame or the previous frame
  • B is the number of effective blocks of the specified frame or previous frame
  • C is the number of effective coefficients of the specified frame or previous frame
  • BR is the bit rate of the specified frame or previous frame
  • Q is the quantization of the specified frame or previous frame
  • the average step size ⁇ Q is the difference between the average quantization step sizes of the predetermined frame and the previous frame, or the average difference between the quantization step sizes of the previous frame and the previous frame
  • I is the predetermined frame power picture.
  • E is the performance required for decoding the previous frame.
  • the amount, P is represents a necessary calculation amount of the previous frame calculated by the required calculation amount calculating means.
  • the function G will be described below.
  • the amount of computation required for decoding a predetermined frame depends on the number of times ID CT processing, IQ processing, and VLD processing are executed in decoding the predetermined frame.
  • the number of executions of IDCT processing depends on the number of effective blocks included in the predetermined frame
  • the number of executions of IQ processing and VLD processing depends on the number of effective coefficients included in the predetermined frame. That is, when the number of effective blocks and the number of effective coefficients included in a predetermined frame is large (small), the amount of calculation required for the decoding process is large (small). Therefore, the above function G is configured so that Kp is set large (small) when B and C are large (small V).
  • the above function G is configured to set Kp small when the predetermined frame is an I picture.
  • the above function G is configured so that Kp is set large (small) when FB and BR are large (small). Also, since the quantization step size is changed when the bit rate is controlled, the above function G can be calculated by taking into consideration the average value Q of the quantization step size and the difference ⁇ Q of the average value of the quantization step size.
  • the Kp calculated by can be a value close to the amount of computation required to actually decode the predetermined frame.
  • the amount of computation required for decoding the predetermined frame is close to the amount of computation E actually required for the decoding processing of the previous frame. It becomes. Furthermore, if the predicted amount of computation calculated by the necessary amount-of-computation calculation means is close to the amount of computation required for the actual decoding process, P ⁇ E. Therefore, considering E and P, the Kp calculated by the above function G is necessary to actually decode the predetermined frame. It can be a value close to the amount of computation.
  • FB is the amount of generated information (number of bits) for one frame.
  • the function G is a function derived by using one or more elements FB, MVa, MVv, B, C, BR, D, Q, ⁇ Qprev, I, E, and P.
  • the required amount of computation Kp is the computation performance (frequency, cycle) that is predicted to be necessary for a given frame. The value is high when the number of bits FB in a given frame is large, and is low when the number of bits FB is small.
  • the required calculation amount Kp when the predetermined frame is the intraframe code ⁇ is small, and the required calculation amount Kp when the predetermined frame is the interframe code ⁇ is a large value.
  • the required calculation amount Kp is the average value of the motion vector size (for the frame to be decoded or the previous frame) MVA, the variance of the size of the motion vector (for the frame to be decoded or the previous frame) MVv, number of effective blocks (for frame to be decoded or previous frame) B, number of effective coefficients (for frame to be decoded or previous frame) C, bit rate ( The frame to be decoded or the previous frame) BR, the amount of generated information (the frame to be decoded or the previous frame) FB, the average quantization step size (this power of the frame to be decoded) Q, the difference between the average quantization step sizes (this is the difference between the Q of the frame to be decoded and the Q of the previous frame) , Or the difference between the Q of the previous frame and the Q of the second previous frame) AQ, I picture power ⁇ Picture power ⁇ Picture type I, actually decoding the previous frame The amount of computation required ⁇ and the predicted value of the amount of computation required for decoding the previous frame
  • the average motion vector size (from the frame to be decoded or from the previous frame) MVa the variance of the motion vector size (from the frame to be decoded or from the previous frame) MVv ,
  • the number of effective blocks (from the frame to be decoded or from the previous frame) B the number of effective coefficients (from the frame to be decoded or from the previous frame) C
  • the value of the element is large.
  • p is relatively large, and the element value is small !, large in some cases, and the required amount of computation Kp is relatively small compared to the case.
  • the necessary calculation amount calculation means 42 only one of these elements may be used or a plurality of elements may be used in combination. In other words, these multiple elements are elements that affect the required amount of calculation necessary for the decoding process of a predetermined frame, and therefore the required calculation amount calculating means 42 needs to calculate the necessary amount of calculation according to these elements. By calculating so as to increase or decrease Kp (cycle), the required calculation amount Kp calculated by the required calculation amount calculating means 42 becomes closer to the calculation amount when the decoding process is actually performed.
  • the operation determining means 3 (operation determining step) and the operation control means 4 are the same as in the first embodiment.
  • the moving picture decoding means 45 decodes the input code key data 401 of a predetermined frame to generate decoded data 406 (moving picture decoding step).
  • the operation control means 4 performs the decoding process while the processor 1 is operated by the operation control means 4 at a constant operating power supply voltage, substrate bias voltage and operating frequency. .
  • the required amount of computation required before decoding the frame is calculated, and the processor is operated while operating at a constant operating frequency, operating power supply voltage, and substrate bias voltage according to the required amount of computation. Since frame decoding is performed, low power consumption can be achieved.
  • the decrypted data 406 is displayed as a moving image on an image display unit of a mobile phone or a personal computer, or stored in a storage medium such as a hard disk.
  • a storage medium such as a hard disk.
  • the moving image decoding process system S4 also includes the third failure avoiding means 97, 98.
  • Each failure avoiding means is almost the same as in the third embodiment, but differs in that it judges not only the amount of computation of the encoding process but the amount of computation of the decoding process.
  • the third failure prevention means 97, 98 can avoid the failure phenomenon.
  • the moving image code processing system of the present invention includes first failure avoiding means 91, 92, 93, 94, first failure avoiding means with mitigation means 95, second failure avoiding means 96, Third bankruptcy avoider
  • the steps 97 and 98 may be provided independently, or each means may be provided in appropriate combination.
  • the decryption processing system may be provided with each second failure avoiding means 96 and third failure avoiding means 97 and 98, or may be provided with an appropriate combination of each means. For example, in the case of a sign key processing system, any one of the first failure avoidance means 91, 92, 93, 94, the first failure avoidance means with the mitigation means 95, and the second failure avoidance means 96 is used.
  • the first failure avoiding means 91 performs invalid block processing only on the color difference block or
  • the intra-frame code processing is performed by the first failure avoiding means 92, or the inter-frame coding process is performed by setting the motion vector size to 0 by the first failure avoiding means 93, or the first failure
  • the avoidance means 94 may increase the quantization step size and perform the code processing.
  • the moving image encoding or decoding processing program may realize low power consumption by controlling the operating frequency, the substrate bias voltage, and the operating voltage in two stages, and has the same function as the program. It may be realized by hardware.
  • the first to fourth embodiments control the operating power supply voltage, the substrate bias voltage, and the operating frequency.
  • the present embodiment controls the operating power supply voltage and the operating frequency. Thus, low power consumption is achieved.
  • this embodiment is not limited to a system integrated in a semiconductor.
  • FIG. 17 is a schematic block diagram showing the operation of the video encoding system S5 of the present embodiment
  • FIG. 18 is a conceptual diagram showing the relationship between the operating power supply voltage and the operating frequency of the processor 51.
  • the moving image coding processing system S5 of this embodiment is variable to the operating power supply voltage Vdd and the operating frequency force Sr stage (r is an integer on 2) instead of the processor 1 of the first embodiment.
  • the operation control means 54 controls the operation power supply voltage and the operation frequency of the processor 1.
  • the operation power supply voltage and the operation frequency of the processor 51, or the processor 1 and peripheral devices are controlled by the operation control means 52.
  • the operation determining unit 53 calculates F (n)> Fe and selects an operation frequency F (n) satisfying Fe> F (n-1) as an operation frequency for performing encoding processing of a predetermined frame.
  • the operation control means 54 sends the values of the operation frequency F (n) and the operation power supply voltage Vdd (n) received from the operation determination means 53 to peripheral devices including the processor 1 and / or the local decoding memory 6 and the like. (Symbol 505), and the processor 1 is controlled at a constant operating frequency F (n) and operating power supply voltage Vdd (n).
  • peripheral devices including the processor 1 and / or the local decoding memory 6 operate at the operating power supply voltage Vdd (n) and the operating frequency F (n).
  • the other points are almost the same as in the first embodiment.
  • the first failure avoiding means 91, 92, 93, 94, 95, the second failure avoiding means 96, or the third failure avoidance Preferably means 97, 98 are provided.
  • the first to fourth embodiments control the operating power supply voltage, the substrate bias voltage, and the operating frequency.
  • the substrate bias voltage and the operating frequency are controlled. By doing so, the power consumption can be reduced.
  • FIG. 19 is a schematic block diagram showing the operation of the moving image encoding system S6 of the present embodiment
  • FIG. 20 is a conceptual diagram showing the relationship between the substrate bias voltage and the operating frequency of the processor 61.
  • the substrate bias voltage Vbn, Vbp and operating frequency power S r step (r is an integer of 2 or more)
  • the plugging device 61 is variable (that is, can operate at r-stage substrate bias voltages Vbn, Vbp and operating frequency) and can change the substrate bias voltage and operating frequency by a program.
  • the operation control means 64 controls the substrate bias voltage and the operating frequency of the processor 1.
  • the processor 61 or the processor 61 and peripheral devices (such as the local decoding memory 6 and the input frame memory 7) have their substrate bias voltage and operating frequency controlled by the operation control means 64.
  • the operation determination unit 63 calculates F (n)> Fe and selects an operation frequency F (n) satisfying Fe> F (n-1) as an operation frequency for performing encoding processing of a predetermined frame. Calculating the substrate bias voltage Vbn (n), Vbp (n) suitable for the operating frequency F (n)! ⁇ , including the processor 1 and / or the local decoding memory 6 etc.
  • the operation control means 64 is instructed to operate the peripheral device at the operating frequency F (n) and the substrate bias voltages Vbn (n) and Vbp (n) (reference numeral 602).
  • the operation control means 64 receives the values of the operating frequency F (n) and the substrate bias voltages Vbn (n) and Vbp (n) received from the operation determining means 63 from the processor 61 and / or the local decoding memory 6 and the like. Supplied to the included peripheral device (reference numeral 605), and controls the processor 61 to operate constantly at its operating frequency F (n) and substrate bias voltages Vbn (n) and Vbp (n). As a result, peripheral devices including the processor 61 and / or the local decoding memory 6 and the like operate at a constant substrate bias voltage Vbn (n), Vbp (n) and an operating frequency F (n). The other points are almost the same as in the first embodiment.
  • the first failure avoiding means 91, 92, 93, 94, 95 or the third failure avoiding means 97, 98, or the second failure avoiding means are provided.
  • the operation power supply voltage suitable for the operation frequency instead of controlling the operation power supply voltage and the substrate bias voltage suitable for the operation frequency, the operation power supply voltage suitable for the operation frequency, Alternatively, the substrate bias voltage can be controlled.
  • the present example is an example of a system in which the moving image coding system 2 of the second embodiment includes the first failure avoiding means 91 and the second failure avoiding means.
  • the 35th frame will be described as an example of the encoded frame.
  • Each frame is composed of a 144 ⁇ 176 pixel array.
  • MPEG-4 is used as the encoding process.
  • FIG. 21 shows an example of the relationship between the operating frequency, the operating power supply voltage, and the substrate bias voltage in the processor 1 of the video encoding system S2.
  • the processor 1 of the video code system S 2 has an operating frequency of 50 MHz to 250 MHz, an operating power supply voltage of 0.5 V to 1. OV,
  • the first voltage is 1.0V to 0.5V, and is variable in 5 steps.
  • the moving image coding system S2 accesses the input frame memory 7, obtains the 35th frame, and calculates the required calculation amount Kp of the frame by the required calculation amount calculation means 2. Specifically, the required amount of computation Kp first calculates the sum of absolute differences Z using the following formula using the 34th frame as the previous frame.
  • the actual number of bits generated in the previous frame, D 56797, is obtained.
  • the required amount of computation Kp is calculated using the following formula using each element.
  • the switching timing time is calculated from the following formula.
  • Ks is the number of cycles required to process only the color difference block as an invalid block in one macroblock.
  • Kw X (MB-MBi) and F X (Te-Ti) are determined, and it is determined that there is a sufficient amount of calculation, and the switching timing is updated.
  • the 35th frame step ends, and the next 36th frame step is entered.
  • the second failure avoiding means 96 is used to predict the required amount of computation. Increase the process.
  • the number M of macroblock matching, the number B of effective blocks, and the number C of effective coefficients are multiplied by 1.1, and the necessary calculation amount Kp is calculated by the following formula.
  • Kp j + XI. L) M + (j8 XI. 1) ⁇ + ( ⁇ XI. L) C + ⁇ + ⁇ AQprev
  • the quantization step size is determined in the macroblock of the predetermined frame corresponding to the macroblock for which the failure avoidance processing has been executed in the previous frame by the first failure avoiding means 95.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

[PROBLÈMES] Proposer un système de codage ou décodage d’image vidéo et un procédé de codage ou décodage d’image vidéo permettant d’éviter le phénomène d’échec lorsqu’une consommation de faible puissance est atteinte. [MOYENS POUR RÉSOUDRE LES PROBLÈMES] La présente invention comprend des moyens de calcul de complexité algorithmique (2) nécessaire pour calculer une complexité algorithmique nécessaire pour coder ou décoder une trame prédéterminée, des moyens de détermination de fonctionnement (3) pour calculer la tension d’alimentation de puissance de fonctionnement, une tension de polarisation de substrat et une fréquence de fonctionnement avec lesquels le codage ou le décodage est possible pendant un temps Te précédemment alloué pour coder ou décoder la trame prédéterminée. Pendant le temps Te, un processeur (1) code ou décode la trame prédéterminée avec la fréquence de fonctionnement constante calculée, la tension d’alimentation de puissance de fonctionnement, et la tension de polarisation de substrat. L’invention comprend en outre des moyens pour éviter l’échec pour évider un phénomène d’échec qui se produit lorsque la complexité algorithmique nécessaire calculée par les moyens de calcul de la complexité algorithmique nécessaire est plus petite que la complexité algorithmique réellement nécessaire.
PCT/JP2005/012250 2004-07-02 2005-07-01 Système de codage d’image vidéo, système de codage ou décodage d’image vidéo, procédé de codage d’image video, et procédé de codage ou décodage d’image video WO2006004065A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2006528865A JPWO2006004065A1 (ja) 2004-07-02 2005-07-01 動画像符号化処理システム、動画像符号化又は復号化処理システム、動画像符号化処理方法、及び、動画像符号化又は復号化処理方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2004197188 2004-07-02
JP2004-197188 2004-07-02

Publications (2)

Publication Number Publication Date
WO2006004065A1 true WO2006004065A1 (fr) 2006-01-12
WO2006004065A8 WO2006004065A8 (fr) 2006-02-23

Family

ID=35782869

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2005/012250 WO2006004065A1 (fr) 2004-07-02 2005-07-01 Système de codage d’image vidéo, système de codage ou décodage d’image vidéo, procédé de codage d’image video, et procédé de codage ou décodage d’image video

Country Status (2)

Country Link
JP (1) JPWO2006004065A1 (fr)
WO (1) WO2006004065A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008104153A (ja) * 2006-08-29 2008-05-01 Nvidia Corp ビデオ復号時に動的周波数調整する方法及び装置
JP2008136177A (ja) * 2006-10-27 2008-06-12 Matsushita Electric Ind Co Ltd 動き検出装置、ＭＯＳ（ｍｅｔａｌ−ｏｘｉｄｅｓｅｍｉｃｏｎｄｕｃｔｏｒ）集積回路および映像システム
JP2008141311A (ja) * 2006-11-30 2008-06-19 Sharp Corp 復号装置および復号方法
US8098898B2 (en) 2006-10-27 2012-01-17 Panasonic Corporation Motion detection device, MOS (metal-oxide semiconductor) integrated circuit, and video system
JP2014220020A (ja) * 2013-04-30 2014-11-20 富士通株式会社 電子回路
CN111342937A (zh) * 2020-03-17 2020-06-26 北京百瑞互联技术有限公司 一种动态调整编解码处理器电压和/或频率的方法和装置

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002232881A (ja) * 2001-01-31 2002-08-16 Toshiba Corp 動画像復号化方法及び装置

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002232881A (ja) * 2001-01-31 2002-08-16 Toshiba Corp 動画像復号化方法及び装置

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
KAWAGUCHI H. ET AL: "An LSI for Vdd-Hopping and MPEG1 System Based on the Chip", IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEM 2001, BEIKOKU, IEEE 2001 NEN 5 GATSU, 2001, pages 918 - 921, XP010542007 *
KAWAKAMI K. ET AL: "Feedfoward-gata Doteki Den'atsu Seigyo ni yoru MPEG4 Teishohi Denryokuka Algorithm", THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS GIJUTSU KENKYU HOKOKU ICD2003-30, NIPPON, THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS, 2003 NEN 5 GATSU, 2003, pages 25 - 30, XP002991854 *
LEE S AND SAKURAI TAKAYASU.: "Run-time Voltage Hopping for Low-power Real-time Systems", PROC. DESIGN AUTOMATION CONFERENCE, BEIKOKU, IEEE 2000 NEN 6 GATSU, 2000, pages 806 - 809, XP002991852 *
MORITA Y. ET AL: "D-23 Feedfoward Seigyo o Mochiita MPEG4 Teishohi Denryokuka Algorithm (2) -2 Dankai Dosa Den'atsu/Shuhasu Seigyoteki Yoji no Shohi Denryoku Sakugen Koka no Mitsumori", HEISEI 15 NENDO RECORD OF JOINT CONFERENCE OF ELECTRICAL AND ELECTRONICS ENGINEERS IN HOKURIKU, NIPPON, HEISEI 15 NENDO INSTITUTES OF ELECTRICAL AND INFORMATION ENGINEERS, JAPAN HOKURIKU SHIBU RENGO TAIKAI JIKKO IINKAI JIMUKYOKU, 22 September 2003 (2003-09-22), pages 192, XP002991855 *
MORITA Y. ET AL: "Dagazo Fugoka Shgori ni Okeru Denryoku Saishoka no tameno Feedfoward-gata Doteki Shuhasu/Den'atsu Algorithm", THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS GIJUTSU KENKYU HOKOKU, ICD2004-26, NIPPON, THE INSTITUTE OF ELECTRONICS, 2004 NEN 5 GATSU, 2004, pages 19 - 24, XP002996675 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008104153A (ja) * 2006-08-29 2008-05-01 Nvidia Corp ビデオ復号時に動的周波数調整する方法及び装置
JP2012075145A (ja) * 2006-08-29 2012-04-12 Nvidia Corp ビデオ復号時に動的周波数調整する方法及び装置
TWI413418B (zh) * 2006-08-29 2013-10-21 Nvidia Corp 於視訊解碼時動態頻率調整之方法、系統、與時鐘頻率控制器
TWI448161B (zh) * 2006-08-29 2014-08-01 Nvidia Corp 於視訊解碼時動態頻率調整之方法、系統、與時鐘頻率控制器
JP2008136177A (ja) * 2006-10-27 2008-06-12 Matsushita Electric Ind Co Ltd 動き検出装置、ＭＯＳ（ｍｅｔａｌ−ｏｘｉｄｅｓｅｍｉｃｏｎｄｕｃｔｏｒ）集積回路および映像システム
US8098898B2 (en) 2006-10-27 2012-01-17 Panasonic Corporation Motion detection device, MOS (metal-oxide semiconductor) integrated circuit, and video system
JP2008141311A (ja) * 2006-11-30 2008-06-19 Sharp Corp 復号装置および復号方法
JP2014220020A (ja) * 2013-04-30 2014-11-20 富士通株式会社 電子回路
CN111342937A (zh) * 2020-03-17 2020-06-26 北京百瑞互联技术有限公司 一种动态调整编解码处理器电压和/或频率的方法和装置
CN111342937B (zh) * 2020-03-17 2022-05-06 北京百瑞互联技术有限公司 一种动态调整编解码处理器电压和/或频率的方法和装置

Also Published As

Publication number Publication date
JPWO2006004065A1 (ja) 2008-07-31
WO2006004065A8 (fr) 2006-02-23

Similar Documents

Publication Publication Date Title
US11425371B2 (en) Intra/inter mode decision for predictive frame encoding
US8660182B2 (en) MPEG motion estimation based on dual start points
US8659607B2 (en) Efficient video decoding migration for multiple graphics processor systems
US20070030899A1 (en) Motion estimation apparatus
US20110216829A1 (en) Enabling delta compression and modification of motion estimation and metadata for rendering images to a remote display
JP3772183B2 (ja) 動画像符号化処理システム、動画像復号化処理システム、動画像符号化処理プログラム、及び、動画像復号化処理プログラム
JP2009055542A (ja) 動画像符号化装置および動画像符号化方法
JPWO2004093458A1 (ja) 動画像符号化又は復号化処理システム及び動画像符号化又は復号化処理方法
JP2005515732A (ja) 計算量制約データ圧縮における動的制御
JPWO2007032123A1 (ja) マルチストリーム復号装置
WO2006004065A1 (fr) Système de codage d’image vidéo, système de codage ou décodage d’image vidéo, procédé de codage d’image video, et procédé de codage ou décodage d’image video
US10694178B2 (en) Video processing device for encoding frames constituting hierarchical structure
TWI543627B (zh) 自適應視頻解碼方法、複雜度自適應視頻解碼器以及自適應音頻／視頻播放系統
JP2011233991A (ja) 符号化装置および符号化方法
US20030133504A1 (en) Image coding integrated circuit capable of reducing power consumption according to data to be processed
WO2005062621A1 (fr) Systeme de codage ou de decodage, procede de codage ou de decodage d'images animees
WO2005096634A1 (fr) Systeme de traitement multi-taches et procede de traitement multi-taches
US20130315296A1 (en) Systems and methods for adaptive selection of video encoding resources
JP2003116136A (ja) 動画像復号装置
JP3772184B2 (ja) 動画像符号化処理システム、動画像復号化処理システム、動画像符号化処理方法、及び、動画像復号化処理方法
US8170356B2 (en) Linear temporal reference scheme having non-reference predictive frames
US8126277B2 (en) Image processing method, image processing apparatus and image pickup apparatus using the same
Hsia et al. A real-time chip implementation for adaptive video coding control
US10063873B2 (en) Method for adaptively performing video decoding, and associated adaptive complexity video decoder and adaptive audio/video playback system
Chiu et al. A lossless embedded compression codec engine for HD video decoding

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

CFP Corrected version of a pamphlet front page
CR1 Correction of entry in section i

Free format text: IN PCT GAZETTE 02/2006 DUE TO A TECHNICAL PROBLEM AT THE TIME OF INTERNATIONAL PUBLICATION, SOME INFORMATION WAS MISSING UNDER (57), THE MISSING INFORMATION NOW APPEARS IN THE CORRECTED VERSION

121 Ep: the epo has been informed by wipo that ep was designated in this application
DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2006528865

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

122 Ep: pct application non-entry in european phase