TWI869992B - Video encoder and video encoding method - Google Patents

Abstract

A video coding method includes the following steps: performing a first optimization operation on a first sub-coding block to select original pixels or reconstructed pixels of a neighboring block of the first sub-coding block according to a base prediction mode to generate an intermediate prediction mode of the first sub-coding block; performing a second optimization operation on the first sub-coding block to determine a prediction mode according to the intermediate prediction mode, the original pixels of the first sub-coding block, and the reconstructed pixels of the neighboring block of the first sub-coded block; generating a prediction information of the first sub-coding block according to the original pixels of the first sub-coded block, the reconstructed pixels of the neighboring block of the first sub-coded block, and the prediction mode; generating an encoding coefficient and reconstructed pixels of the first sub-coding block according to the original pixels of the first sub-coding block and the prediction information; and generating a bit stream according to the encoding coefficient and the prediction information.

Description

Video encoder and video encoding method

The present invention relates to video coding, and more particularly to a video codec and a video coding method.

In the field of video coding, fast rate-distortion optimization (RDO) and full rate-distortion optimization (full RDO) are commonly used for mode prediction. Full RDO corresponds to the conventional real/standard, which refers to the Lagrange rate-distortion cost (RD cost) evaluation method (as shown in the following equation (1)): J = arg min ( D + λR ) (1)

Among them, J is the rate-distortion cost of a certain mode; D is the distortion corresponding to the mode; R is the rate cost corresponding to the mode, including the mode syntax cost and the coefficient syntax cost; λ is the coefficient corresponding to the current quantization parameter.

The complete rate-distortion optimization calculates equation (1) for each candidate mode and selects the mode corresponding to the minimum J as the best mode. However, this consumes a lot of hardware resources and time.

Fast rate-distortion optimization is calculated according to equation (2). Because R' only includes the pattern syntax cost but not the coefficient syntax cost, it can significantly save hardware resources and time.

J = arg min ( D + λR' ) (2)

However, if fast rate-distortion optimization is performed on original pixels rather than reconstructed pixels, the accuracy is not high. Reconstructed pixels are the result of prediction, residual calculation, transformation, quantization, inverse quantization, inverse transformation, and reconstruction (sometimes including filtering) of original pixels.

Therefore, a video encoder and method are needed to improve speed and accuracy.

In view of the shortcomings of the prior art, one purpose of the present invention is to provide a video encoder and a video encoding method to improve the shortcomings of the prior art.

An embodiment of the present invention provides a video encoder, which includes a prediction circuit, a calculation circuit and a coding circuit. The prediction circuit is configured to: perform a first optimization operation on a first sub-coding block to select one of original pixels or reconstructed pixels of a neighboring block of the first sub-coding block according to a basic prediction mode to generate an intermediate prediction mode of the first sub-coding block; perform a second optimization operation on the first sub-coding block to determine a prediction mode according to the intermediate prediction mode, the original pixels of the first sub-coding block and the reconstructed pixels of the neighboring block of the first sub-coding block; and generate prediction information of the first sub-coding block according to the original pixels of the first sub-coding block, the reconstructed pixels of the neighboring block of the first sub-coding block and the prediction mode, the prediction information comprising prediction pixels and residual values between the original pixels and the prediction pixels. The calculation circuit is coupled to the prediction circuit, and is used to generate a coding coefficient and reconstructed pixel of the first sub-coding block according to the prediction information of the first sub-coding block. The coding circuit is coupled to the prediction circuit and the calculation circuit, and is used to generate a code stream according to the coding coefficient and the prediction information.

Another embodiment of the present invention provides a video encoding method, comprising: performing a first optimization operation on a first sub-coding block to select one of original pixels or reconstructed pixels of a neighboring block of the first sub-coding block according to a basic prediction mode to generate an intermediate prediction mode of the first sub-coding block; performing a second optimization operation on the first sub-coding block to select one of original pixels or reconstructed pixels of a neighboring block of the first sub-coding block according to the intermediate prediction mode, the original pixels of the first sub-coding block and the neighboring block of the first sub-coding block. A prediction mode is determined based on the reconstructed pixels of the first sub-coding block; prediction information of the first sub-coding block is generated based on the original pixels of the first sub-coding block, the reconstructed pixels of the adjacent block of the first sub-coding block and the prediction mode, wherein the prediction information includes the prediction pixel and the residual value between the original pixel and the prediction pixel; a coding coefficient and reconstructed pixels of the first sub-coding block are generated based on the prediction information of the first sub-coding block; and a bitstream is generated based on the coding coefficient and the prediction information.

The technical means embodied in the embodiments of the present invention can improve at least one of the shortcomings of the previous technology, so the present invention can improve the prediction speed and accuracy compared to the previous technology.

The features, implementation and effects of the present invention are described in detail below with reference to the accompanying drawings.

100: Electronic devices

110: External memory

120: Video encoder

121: Direct memory access circuit

122: Memory

122A, 122B: memory block

123: Prediction circuit

124: Computing circuit

125: Encoding circuit

210: Mode determination circuit

Bts: bitstream

Coe: Coding coefficient

Din: Video data

PI: Predicted Information

PI_intra: Intra-frame prediction information

PI_mo: Sports information

PM: Prediction model

Rec: Reconstructed pixels

Src: original pixel

212: Preliminary mode judgment circuit

214: Fast bit rate-distortion optimization circuit

216: Complete bit rate-distortion optimization circuit

BM: Basic forecasting model

IM: Intermediate Forecast Model

B(1,2),B(1,3),B(2,3),B(3,1),B(x,y),B(2,1),B(2,2): coding block

bs(1,1),bs(1,2),bs(2,1),bs(2,2),bs(3,1),bs(3,2),bs(3,3),bs(3,4),bs(4,1),bs(4,2),bs(4,3),bs(4,4),bs16_0~bs16_3,bs8_0~bs8_15,bs4_0~bs4_63: sub-coding block

BSL1: First-level coding block

HCL: Horizontal Center Line

LfB:Left Boundary

LwB: Lower Boundary

RtB:Right Boundary

UpB: upper boundary

VCL: Vertical Center Line

T1,T2,T3,T4,T5,T6: time points

BSL2: Second level coding block

BSL3: Third level coding block

BSL4: Fourth level coding block

610,S620,S630,S640,S650,S710,S712,S714,S720,S722,S724,S726,S730,S732,S734,S736,S738,S740,S742,S744,S750,S752,S754,S756,S810,S820,S830,S840: Steps

FIG. 1 is a functional block diagram of an embodiment of the electronic device of the present invention; FIG. 2 is a functional block diagram of an embodiment of the mode determination circuit of the present invention; FIG. 3 is a schematic diagram of an embodiment of the frame of the present invention; FIG. 4 is a schematic diagram of an embodiment of the operation timing of the sub-coding block of the present invention; FIG. 5 is a schematic diagram of a higher level of the coding block of the present invention; FIG. 6 is a flow chart of an embodiment of the video coding method of the present invention; FIG. 7A to FIG. 7E are partial flow charts of an embodiment of the first optimization operation of the present invention; and FIG. 8 is a partial flow chart of another embodiment of the first optimization operation of the present invention.

The technical terms used in the following descriptions refer to the customary terms in this technical field. If this manual explains or defines some of the terms, the interpretation of those terms shall be based on the explanation or definition in this manual.

The disclosure of the present invention includes a video encoder and a video encoding method. Since some components included in the video encoder of the present invention may be known components individually, the details of the known components will be omitted in the following description without affecting the full disclosure and feasibility of the device invention. In addition, part or all of the process of the video encoding method of the present invention may be in the form of software and/or firmware, and can be executed by the video encoder of the present invention or its equivalent device. Without affecting the full disclosure and feasibility of the method invention, the following description of the method invention will focus on the step content rather than the hardware.

Please refer to FIG. 1, which is a functional block diagram of an embodiment of the electronic device of the present invention. The electronic device 100 includes an external memory 110 and a video encoder 120. The video encoder 120 includes a direct memory access (DMA) circuit 121, a memory 122, a prediction circuit 123, a calculation circuit 124, and a coding circuit 125. The memory 122 includes a memory block 122A and a memory block 122B. The prediction circuit 123 includes a mode determination circuit 210.

The external memory 110 stores the video data Din. The direct memory access circuit 121 reads the original pixel Src in the video data Din from the external memory 110 and stores the original pixel Src in the memory block 122A. The memory block 122B stores the reconstructed pixel Rec generated by the calculation circuit 124.

The prediction circuit 123 generates prediction information PI according to the prediction mode PM, the original pixel Src and the reconstructed pixel Rec of the reference block. The prediction information PI includes inter-frame prediction information such as motion information PI_mo and intra-frame prediction information PI_intra. The calculation circuit 124 generates the reconstructed pixel Rec and the coding coefficient Coe according to the prediction information PI. The coding circuit 125 generates the bit stream Bts according to the prediction information PI and the coding coefficient Coe.

The operations performed by the computing circuit 124 include transformation, quantization, inverse quantization, inverse transformation and filtering. The operating principles of the computing circuit 124 and the encoding circuit 125 are well known to those with ordinary knowledge in the technical field, so they will not be elaborated.

Please refer to FIG. 2, which is a functional block diagram of an embodiment of the mode determination circuit of the present invention. The mode determination circuit 210 is a part of the prediction circuit 123, including a rough mode determination circuit 212, a fast bit rate-distortion optimization circuit 214 and a complete bit rate-distortion optimization circuit 216.

The preliminary mode determination circuit 212 selects a basic prediction mode BM from 8 main directions. The fast bit rate-distortion optimization circuit 214 determines 6 directions according to the direction corresponding to the basic prediction mode BM, and then determines the intermediate prediction mode IM from the direction corresponding to the basic prediction mode BM, the 6 directions and the other 3 preset directions (a total of 10 modes). The complete bit rate-distortion optimization circuit 216 determines the prediction mode PM according to the intermediate prediction mode IM. In the process of generating the intermediate prediction mode IM, the fast bit rate-distortion optimization circuit 214 uses the original pixel Src or the reconstructed pixel Rec to perform the first optimization operation. In the process of generating the prediction mode PM, the complete bit rate-distortion optimization circuit 216 uses the reconstructed pixel Rec to perform the second optimization operation.

More specifically, for the I frame, the fast rate-distortion optimization circuit 214 selects 2 from the 10 modes, and then the complete rate-distortion optimization circuit 216 determines the final best mode of intra-frame prediction (i.e., prediction mode PM) from the 2 modes. For the P frame, the fast rate-distortion optimization circuit 214 performs intra-frame prediction to select a better intra-frame mode from the 10 modes, and performs inter-frame prediction to obtain a better inter-frame mode, and then the complete rate-distortion optimization circuit 216 compares the better intra-frame mode with the better inter-frame mode to generate the prediction mode PM.

Please refer to Figure 3, which is a schematic diagram of an embodiment of a frame of the present invention. A frame includes a plurality of coding blocks (B(1,1), B(1,2), B(1,3), ..., B(2,1), B(2,2), B(2,3), ..., B(3,1), ...), each coding block includes a plurality of first-level sub-coding blocks BSL1. For example, the first-order sub-coding block BSL1 of coding block B(1,1) includes sub-coding blocks bs(1,1), bs(1,2), bs(2,1) and bs(2,2); the first-order sub-coding block BSL1 of coding block B(2,1) includes sub-coding blocks bs(3,1), bs(3,2), bs(4,1) and bs(4,2); the first-order sub-coding block BSL1 of coding block B(2,2) includes sub-coding blocks bs(3,3), bs(3,4), bs(4,3) and bs(4,4). B(1,1), B(1,2), B(1,3) are arranged consecutively, and B(2,1), B(2,2), B(2,3) are arranged consecutively.

The video encoder 120 processes the coding blocks in order from left to right and from top to bottom (i.e., according to the following order: B(1,1)→B(1,2)→B(1,3)→…→B(2,1)→B(2,2)→B(2,3)→…→B(3,1)→…), and processes the sub-coding blocks in a "Z" order (i.e., taking the coding blocks in the second row as an example, according to the following order: bs(3,1)→bs(3,2)→bs(4,1)→bs(4,2)→bs(3,3)→bs(3,4)→bs(4,3)→bs(4,4)→…).

Each coding block (see the coding block B(x,y) in the lower right corner) has an upper boundary UpB (or top), a lower boundary LwB (or bottom), a left boundary LfB (or left), and a right boundary RtB (or right). Each coding block is divided equally by a horizontal center line HCL and a vertical center line VCL. The horizontal center line HCL is parallel to the upper boundary UpB and the lower boundary LwB, while the vertical center line VCL is parallel to the left boundary LfB and the right boundary RtB.

Please refer to FIG. 4, which is a schematic diagram of an embodiment of the operation timing of the sub-coding block of the present invention. FIG. 4 takes coding block B(2,1) and coding block B(2,2) as examples. The video encoder 120 first performs the first stage operation and then the second stage operation on a certain sub-coding block (for example, sub-coding block bs(3,1)), and simultaneously performs the first stage operation and the second stage operation on two sub-coding blocks that are consecutive in the processing sequence (for example, performing the second stage operation on sub-coding block bs(3,1) while performing the first stage operation on sub-coding block bs(3,2)). The operation of the first stage includes but is not limited to the fast bit rate-distortion optimization circuit 214 generating an intermediate prediction mode IM according to the basic prediction mode BM. The operation of the second stage includes but is not limited to the complete bit rate-distortion optimization circuit 216 generating a prediction mode PM according to the intermediate prediction mode IM.

For example, between time point T5 and time point T6, when the complete bit rate-distortion optimization circuit 216 is performing the second stage operation on the sub-coding block bs(4,2), the fast bit rate-distortion optimization circuit 214 is performing the first stage operation on the sub-coding block bs(3,3).

It should be noted that before the second phase of a sub-coding block ends, the reconstructed pixels Rec corresponding to the sub-coding block will not be completely generated. For example, the reconstructed pixels Rec of the sub-coding block bs(4,2) will not be completely generated until time point T6.

Please refer to FIG. 5, which is a schematic diagram of a coding block of the present invention at a higher level. Assume here that the size of a coding block in FIG. 3 (e.g., coding block B(2,2)) is 64*64 pixels, then the size of any first-level sub-coding block BSL1 of the coding block (e.g., sub-coding block bs(3,3)) is 32*32 pixels, and includes 4 sub-coding blocks of 16*16 pixels (i.e., bs16_0, bs16_1, bs16_2, bs16_3, collectively referred to as the second-level sub-coding block BS of the coding block). L2), 16 sub-coding blocks of 8*8 pixels (i.e., bs8_0, bs8_1, bs8_2, ..., bs8_14, bs8_15, collectively referred to as the third-order sub-coding block BSL3 of the coding block), and 64 sub-coding blocks of 4*4 pixels (i.e., bs4_0, bs4_1, bs4_2, ..., bs4_62, bs4_63, collectively referred to as the fourth-order sub-coding block BSL4 of the coding block).

Please continue to refer to FIG5. The sub-coding block bs16_0 includes the sub-coding block bs8_0, the sub-coding block bs8_1, the sub-coding block bs8_2 and the sub-coding block bs8_3; the sub-coding block bs16_1 includes the sub-coding block bs8_4, the sub-coding block bs8_5, the sub-coding block bs8_6 and the sub-coding block bs87; and so on. Subcoding block bs8_0 includes subcoding block bs4_0, subcoding block bs41, subcoding block bs4_2 and subcoding block bs4_3; subcoding block bs8_1 includes subcoding block bs4_4, subcoding block bs4_5, subcoding block bs4_6 and subcoding block bs4_7; and so on.

It should be noted that the video encoder 120 performs encoding operations on each level (i.e., the first-level sub-coding block BSL1, the second-level sub-coding block BSL2, the third-level sub-coding block BSL3, and the fourth-level sub-coding block BSL4) separately. In each level, the video encoder 120 processes the sub-coding blocks in sequence (i.e., in the order shown by the arrows in the figure) according to the number of the sub-coding blocks (i.e., P in bsP_Q is 16, 8, or 4, and Q is 0~3, 0~15, or 0~63).

Please refer to Figure 6, which is a flow chart of an embodiment of the video encoding method of the present invention, which includes the following steps.

Step S610: The fast bit rate-distortion optimization circuit 214 performs a first optimization operation on the sub-coding block to select the original pixels or reconstructed pixels of a neighboring block of the sub-coding block according to the basic prediction mode BM to generate the intermediate prediction mode IM of the sub-coding block. For example (see FIG. 3 ), if the sub-coding block is bs(3,3), the pixels used by the fast bit rate-distortion optimization circuit 214 in this step include the original pixels Src of the sub-coding block bs(3,3) itself and the original pixels Src or reconstructed pixels Rec of the neighboring block (e.g., the sub-coding block bs(4,2), which can be regarded as the reference sub-coding block of the sub-coding block bs(3,3)).

Step S620: The complete rate-distortion optimization circuit 216 performs a second optimization operation on the sub-coding block to determine the prediction mode PM according to the intermediate prediction mode IM, the original pixels Src of the sub-coding block, and the reconstructed pixels Rec of the adjacent blocks of the sub-coding block. For example (see FIG. 3 ), if the sub-coding block is bs(3,3), the pixels used by the complete rate-distortion optimization circuit 216 in this step include the original pixels Src of the sub-coding block bs(3,3) itself and the reconstructed pixels Rec of the adjacent blocks (e.g., the sub-coding block bs(4,2)), but the complete rate-distortion optimization circuit 216 may not need the original pixels Src of the adjacent blocks.

Step S630: The prediction circuit 123 generates prediction information PI according to the original pixel Src of the sub-coding block, the reconstructed pixel Rec of the adjacent block of the sub-coding block and the prediction mode PM. The prediction information PI includes the prediction pixel Prc in the intra-frame prediction mode and the residual value RV between the original pixel Src and the prediction pixel Prc. It is well known to those skilled in the art that the prediction pixel Prc is generated based on the reconstructed pixel Rec in the prediction mode PM.

Step S640: The calculation circuit 124 generates the coding coefficient Coe and reconstructed pixel Rec of the sub-coding block according to the prediction information PI of the sub-coding block.

Step S650: The coding circuit 125 generates a bit stream Bts according to the coding coefficient Coe and the prediction information PI.

Please refer to Figures 7A to 7E, which are partial flow charts of one embodiment of the first optimization operation of the present invention. In the following description, it is assumed that the coding block B (2, 2) is the target coding block.

Please refer to Figure 7A, which contains the following steps.

Step S710: The fast bit rate-distortion optimization circuit 214 determines a target sub-coding block from a plurality of sub-coding blocks of a target coding block. Please refer to FIG. 3 and FIG. 5. For example, the target subcoding block may be one of the first-order subcoding blocks BSL1 (i.e., one of subcoding blocks bs(3,3), bs(3,4), bs(4,3) and bs(4,4)), one of the second-order subcoding blocks BSL2 (i.e., bs16_k, 0≦k≦3), one of the third-order subcoding blocks BSL3 (i.e., bs8_k, 0≦k≦15) or one of the fourth-order subcoding blocks BSL4 (i.e., bs4_k, 0≦k≦63).

Step S712: The fast bit rate-distortion optimization circuit 214 determines whether the current prediction direction is upward. If yes, execute the step of FIG. 7B; otherwise, execute step S714.

Step S714: The fast bit rate-distortion optimization circuit 214 determines whether the current prediction direction is the upper right, left, or lower left. If the current prediction direction is the upper right, the step of FIG. 7C is executed; if the current prediction direction is the left, the step of FIG. 7D is executed; if the current prediction direction is the lower left, the step of FIG. 7E is executed.

Please refer to Figure 7B, which includes the following steps.

Step S720: The fast bit rate-distortion optimization circuit 214 determines whether the target sub-coding block is adjacent to the upper boundary of the target coding block. When the target sub-coding block is adjacent to the upper boundary (lower boundary, left boundary, right boundary) of the target coding block, it means that the upper boundary (lower boundary, left boundary, right boundary) of the target sub-coding block and the upper boundary (lower boundary, left boundary, right boundary) of the target coding block are substantially overlapped, or it means that the upper boundary (lower boundary, left boundary, right boundary) of the target sub-coding block is a part of the upper boundary (lower boundary, left boundary, right boundary) of the target coding block.

Please refer to Figures 3 and 5. For coding block B(2,2), when the target subcoding block is subcoding block bs(3,3), subcoding block bs(3,4), subcoding block bs16_0, subcoding block bs16_1, subcoding block bs8_0, subcoding block bs8_1, subcoding block bs8_4, subcoding block bs8_5, subcoding block bs4_0, subcoding block bs4_1, When the target subcoding block is one of the subcoding blocks bs4_4, bs4_5, bs4_16, bs4_17, bs4_20, and bs4_21, the result of step S720 is yes (go to step S724); when the target subcoding block is another subcoding block, the result of step S720 is no (go to step S722).

Step S722: The fast bit rate-distortion optimization circuit 214 determines whether the upper boundary of the target sub-coding block is adjacent to the horizontal center line HCL of the target coding block. Please refer to FIG3 , the upper boundaries of the sub-coding block bs(3,3) and the sub-coding block bs(3,4) are not adjacent to the horizontal center line HCL, while the upper boundaries of the sub-coding block bs(4,3) and the sub-coding block bs(4,4) are adjacent to the horizontal center line HCL. Please refer to FIG. 5. For subcoding block bs(4,3) and subcoding block bs(4,4), the upper boundaries of subcoding block bs16_0, subcoding block bs16_1, subcoding block bs8_0, subcoding block bs8_1, subcoding block bs8_4, subcoding block bs8_5, subcoding block bs4_0, subcoding block bs4_1, subcoding block bs4_4, subcoding block bs4_5, subcoding block bs4_16, subcoding block bs4_17, subcoding block bs4_20, and subcoding block bs4_21 are adjacent to the horizontal center line HCL.

Step S724: The fast bit rate-distortion optimization circuit 214 uses the reconstructed pixels Rec of the neighboring block above the target sub-coding block for prediction. For the first case (the result of step S720 is yes), since the second phase operation of the neighboring block above the sub-coding block bs(3,3) and the sub-coding block bs(3,4) (coding block B(1,2)) has been completed, the fast bit rate-distortion optimization circuit 214 can use the reconstructed pixels Rec of the neighboring block for prediction. For the second case (the result of step S722 is yes), since the second stage operation of the sub-coding block bs(4,3) and the adjacent blocks above the sub-coding block bs(4,4) (respectively, the sub-coding block bs(3,3) and the sub-coding block bs(3,4)) has been completed, the fast bit rate-distortion optimization circuit 214 can use the reconstructed pixels Rec of the adjacent blocks for prediction.

In some embodiments, the number of reconstructed pixels Rec required by the fast bit rate-distortion optimization circuit 214 when performing the first optimization is related to the size of the target sub-coding block, and the required reconstructed pixels Rec are adjacent to the boundary of the target sub-coding block. For example, when the size of the target sub-coding block is 32*32 (16*16, 8*8, 4*4) pixels, the fast bit rate-distortion optimization circuit 214 requires 32 (16, 8, 4) reconstructed pixels Rec, and the 32 (16, 8, 4) reconstructed pixels Rec are adjacent to the boundary of the target sub-coding block. In some embodiments, the fast bit rate-distortion optimization circuit 214 reads the required reconstructed pixel Rec from the line buffer of the memory 122 (not shown, for example, a portion of the memory block 122B).

Step S726: The fast bit rate-distortion optimization circuit 214 uses the original pixel Src of the neighboring block above the target sub-coding block for prediction. When the target sub-coding block is not adjacent to the first-stage sub-coding block BSL1 that has completed the second-stage operation in the prediction direction, the fast bit rate-distortion optimization circuit 214 can only use the original pixel Src of the neighboring block for prediction. In other words, when the fast bit rate-distortion optimization circuit 214 cannot obtain the required reconstructed pixel Rec from the neighboring block, the fast bit rate-distortion optimization circuit 214 uses the original pixel Src of the neighboring block for prediction instead.

Please refer to Figure 7C, which includes the following steps.

Step S730: The fast bit rate-distortion optimization circuit 214 determines whether the target sub-coding block is adjacent to the upper boundary of the target coding block. This step is the same as step S720. If the result of step S730 is yes, the fast bit rate-distortion optimization circuit 214 executes step S732; otherwise, the fast bit rate-distortion optimization circuit 214 executes step S734.

Step S732: The fast bit rate-distortion optimization circuit 214 uses the reconstructed pixels Rec of the neighboring block to the upper right of the target sub-coding block for prediction. Please refer to the description of step S724. For example, because the second-stage operation of the coding block above (coding block B(1,2)) and the coding block to the upper right (coding block B(1,3)) of the target coding block (coding block B(2,2)) has been completed, the fast bit rate-distortion optimization circuit 214 can use the reconstructed pixels Rec of coding block B(1,2) and coding block B(1,3) for prediction.

Step S734: The fast bit rate-distortion optimization circuit 214 determines whether the upper boundary of the target sub-coding block is adjacent to the horizontal center line HCL of the target coding block. Please refer to the description of step S722.

Step S736: The fast bit rate-distortion optimization circuit 214 determines whether the right boundary of the target sub-coding block is adjacent to the right boundary of the target coding block or the vertical center line VCL. Please refer to Figures 3 and 5. For example, for the sub-coding block bs(4,3) and its sub-coding block bs16_1, sub-coding block bs8_5 and sub-coding block bs4_21, since their right boundaries are adjacent to the vertical center line VCL of the target coding block, the result of step S736 is yes; for the sub-coding block bs(4,4) and its sub-coding block bs16_1, sub-coding block bs8_5 and sub-coding block bs4_21, since their right boundaries are adjacent to the right boundary RtB of the target coding block, the result of step S736 is yes. For other sub-coding blocks whose upper boundaries are adjacent to the horizontal center line HCL, the result of step S736 is no.

Step S738: The fast bit rate-distortion optimization circuit 214 uses the original pixel Src of the upper right neighboring block of the target sub-coding block for prediction. Please refer to the description of step S726. For example, for the sub-coding block bs16_1 of the sub-coding block bs(4,3) (or the sub-coding block bs(4,4)), because the corresponding sub-coding block (i.e., the adjacent block) to the upper right is part of the sub-coding block bs(3,4) (or the coding block B(2,3)), and the sub-coding block bs(3,4) (or the coding block B(2,3)) has not yet completed (or started) the second stage operation, the fast bit rate-distortion optimization circuit 214 now uses the original pixel Src of the adjacent block (i.e., the sub-coding block bs(3,4) (or the coding block B(2,3))) for prediction.

Please refer to Figure 7D, which contains the following steps.

Step S740: The fast bit rate-distortion optimization circuit 214 determines whether the target sub-coding block is adjacent to the left boundary of the target coding block. Referring to FIG. 3 , sub-coding block bs(3,3) and sub-coding block bs(4,3) are adjacent to the left boundary of coding block B(2,2), while sub-coding block bs(3,4) and sub-coding block bs(4,4) are not adjacent to the left boundary of coding block B(2,2). Please refer to Figure 5. For sub-coding block bs(3,3) and sub-coding block bs(4,3), sub-coding block bs16_0, sub-coding block bs16_2, sub-coding block bs8_0, sub-coding block bs8_2, sub-coding block bs8_8, sub-coding block bs8_10, sub-coding block bs4_0, sub-coding block bs4_2, sub-coding block bs4_8, sub-coding block bs4_10, sub-coding block bs4_32, sub-coding block bs4_34, sub-coding block bs4_40 and sub-coding block bs4_42 are adjacent to the left boundary of coding block B(2,2).

Step S742: The fast bit rate-distortion optimization circuit 214 uses the reconstructed pixels Rec of the neighboring block to the left of the target sub-coding block for prediction. Please refer to the description of step S724. For example, when the fast bit rate-distortion optimization circuit 214 processes the sub-coding block bs(3,3) (or the sub-coding block bs(4,3)), the second stage operation of the corresponding sub-coding block bs(3,2) (or the sub-coding block bs(4,2)) on the left (i.e., the adjacent block) has been completed, so the fast bit rate-distortion optimization circuit 214 can use the reconstructed pixel Rec of the adjacent block (i.e., the sub-coding block bs(3,2) (or the sub-coding block bs(4,2))) for prediction.

Step S744: The fast bit rate-distortion optimization circuit 214 uses the original pixel Src of the neighboring block to the left of the target sub-coding block for prediction. When the fast bit rate-distortion optimization circuit 214 cannot obtain the required reconstructed pixel Rec, the fast bit rate-distortion optimization circuit 214 uses the original pixel Src of the neighboring block for prediction instead.

Please refer to Figure 7E, which contains the following steps.

Step S750: The fast bit rate-distortion optimization circuit 214 determines whether the target sub-coding block is adjacent to the left boundary of the target coding block. Please refer to the description of step S740.

Step S752: The fast bit rate-distortion optimization circuit 214 determines whether the lower boundary of the target sub-coding block is adjacent to the lower boundary of the target coding block or the horizontal center line HCL. Please refer to FIG. 3 and FIG. 5. For example, for sub-coding block bs(3,3) and its sub-coding block bs16_2, sub-coding block bs8_10 and sub-coding block bs4_42, since their lower boundaries are adjacent to the horizontal center line HCL of the target coding block, the result of step S752 is yes; for sub-coding block bs(4,3) and its sub-coding block bs16_2, sub-coding block bs8_10 and sub-coding block bs4_42, since their lower boundaries are adjacent to the lower boundary of the target coding block, the result of step S752 is yes. For other sub-coding blocks adjacent to the left boundary of coding block B(2,2), the result of step S752 is no.

Step S754: The fast bit rate-distortion optimization circuit 214 uses the original pixel Src of the neighboring block at the lower left of the target sub-coding block for prediction. Please refer to the description of step S726. For example, for the sub-coding block bs16_2 of the sub-coding block bs(3,3) (or the sub-coding block bs(4,3)), since the corresponding sub-coding block (i.e., the adjacent block) at the lower left is part of the sub-coding block bs(4,2) (or the coding block B(3,1)), and the sub-coding block bs(4,2) (or the coding block B(3,1)) has not yet completed (or started) the second stage operation, the fast bit rate-distortion optimization circuit 214 now uses the original pixel Src of the adjacent block for prediction.

Step S756: The fast bit rate-distortion optimization circuit 214 uses the reconstructed pixels Rec of the neighboring block at the lower left of the target sub-coding block for prediction. Please refer to the description of step S724. For example, when the fast bit rate-distortion optimization circuit 214 processes the sub-coding block bs16_0 of the sub-coding block bs(3,3) (or the sub-coding block bs(4,3)), the corresponding sub-coding block to the lower left (i.e., the neighboring block) is part of the sub-coding block bs(3,2) (or the sub-coding block bs(4,2)), and the second stage operation of the sub-coding block bs(3,2) (or the sub-coding block bs(4,2)) has been completed, so the fast bit rate-distortion optimization circuit 214 can use the reconstructed pixel Rec of the neighboring block for prediction.

In summary, the fast bit rate-distortion optimization circuit 214 can determine whether to use the reconstructed pixel Rec or the original pixel Src for prediction according to the prediction direction and the relative position between the target sub-coding block and the target coding block. Compared with the conventional technology that only uses the original pixel Src for prediction, the present invention can improve the speed and accuracy of prediction.

Please refer to Figure 8, which is a partial flow chart of another embodiment of the first optimization operation of the present invention. The processes of Figures 7A to 7E can be summarized into the process of Figure 8. The process of Figure 8 includes the following steps.

Step S810: The fast bit rate-distortion optimization circuit 214 determines a target sub-coding block from a plurality of sub-coding blocks of a target coding block. Please refer to the description of step S710.

Step S820: The fast bit rate-distortion optimization circuit 214 determines whether a reconstructed pixel corresponding to a reference sub-coding block in the prediction direction (e.g., a neighboring block of the target sub-coding block in the prediction direction) has been generated (i.e., determines whether the reconstructed pixel of the reference sub-coding block has been stored in the memory 122). For example, referring to FIG. 3 , when the target sub-coding block is sub-coding block bs(4,3) (or sub-coding block bs(4,4)) and the prediction direction is upward, the neighboring block of sub-coding block bs(4,3) (or sub-coding block bs(4,4)) in the prediction direction is sub-coding block bs(3,3) (or sub-coding block bs(3,4)). The result of step S820 is yes, which corresponds to the results of step S720, step S722, step S730 and step S740 being yes, and the results of step S736 and step S752 being no. The result of step S820 is no, which corresponds to the results of step S722, step S734, step S740 and step S750 being no, and the results of step S736 and step S752 being yes.

Step S830: The fast bit rate-distortion optimization circuit 214 uses the reconstructed pixels Rec of the adjacent blocks for prediction. Please refer to the description of step S724, step S732, step S742 or step S756.

Step S840: The fast bit rate-distortion optimization circuit 214 uses the original pixel Src of the adjacent block for prediction. Please refer to the description of step S726, step S738, step S744 or step S754.

Although the above-mentioned embodiment takes a 64*64 pixel coding block as an example, this is not a limitation of the present invention. People skilled in the art can appropriately apply the present invention to coding blocks of other sizes according to the disclosure of the present invention.

Although the embodiments of the present invention are described above, these embodiments are not intended to limit the present invention. Those with ordinary knowledge in the technical field may make changes to the technical features of the present invention based on the explicit or implicit content of the present invention. All these changes may fall within the scope of patent protection sought by the present invention. In other words, the scope of patent protection of the present invention shall be subject to the scope of the patent application defined in this specification.

S810, S820, S830, S840: Steps

Claims (12)

A video encoder comprises: a prediction circuit configured to: perform a first optimization operation on a first sub-coding block to select one of original pixels or reconstructed pixels of an adjacent block of the first sub-coding block according to a basic prediction mode to generate an intermediate prediction mode of the first sub-coding block, wherein when the reconstructed pixels of the adjacent block are not generated, the original pixels of the adjacent block are selected to generate the intermediate prediction mode; perform a second optimization operation on the first sub-coding block to select one of original pixels or reconstructed pixels of an adjacent block of the first sub-coding block according to a basic prediction mode to generate an intermediate prediction mode of the first sub-coding block; A prediction mode is determined based on the reconstructed pixels of the first sub-coding block; and prediction information of the first sub-coding block is generated based on the original pixels of the first sub-coding block, the reconstructed pixels of the adjacent block of the first sub-coding block and the prediction mode, wherein the prediction information includes the prediction pixel and the residual value between the original pixel and the prediction pixel; a calculation circuit is coupled to the prediction circuit and used to generate a coding coefficient and reconstructed pixels of the first sub-coding block based on the prediction information of the first sub-coding block; and a coding circuit is coupled to the prediction circuit and the calculation circuit and used to generate a bit stream based on the coding coefficient and the prediction information. A video encoder as claimed in claim 1, wherein the prediction circuit performs the first optimization operation on a second sub-coding block while performing the second optimization operation on the first sub-coding block, and the first sub-coding block and the second sub-coding block are continuous in coding order. The video encoder of claim 2, wherein the first sub-coding block is a neighboring block of the second sub-coding block in a prediction direction of the basic prediction mode, and the first optimization operation is performed on the second sub-coding block according to the original pixels of the first sub-coding block. As in claim 1, the video encoder sequentially processes a first coding block, a second coding block and a third coding block arranged continuously in a frame, the second coding block is located between the first coding block and the third coding block, the second coding block includes a plurality of sub-coding blocks, the first sub-coding block is one of the plurality of sub-coding blocks, and the first optimization operation includes the following steps: according to a prediction direction and a position of the first sub-coding block relative to the second coding block, determine to use the original pixels or reconstructed pixels of the adjacent block of the first sub-coding block to generate the intermediate prediction mode. The video encoder of claim 4, wherein the second coding block includes a first upper boundary, a lower boundary, a left boundary, a right boundary, and a horizontal center line that divides the second coding block equally and is parallel to the first upper boundary. When the first sub-coding block is adjacent to the first upper boundary and the prediction direction is upward, or when a second upper boundary of the first sub-coding block is adjacent to the horizontal center line and the prediction direction is upward, the prediction circuit uses the reconstructed pixels of the adjacent block to generate the intermediate prediction mode. The video encoder of claim 4, wherein the second coding block includes an upper boundary, a lower boundary, a left boundary and a right boundary, and when the first sub-coding block is adjacent to the upper boundary and the prediction direction is the upper right or when the first sub-coding block is adjacent to the left boundary and the prediction direction is the left, the prediction circuit uses the reconstructed pixels of the adjacent block to generate the intermediate prediction mode. The video encoder of claim 4, wherein the second coding block includes a first upper boundary, a lower boundary, a left boundary, a first right boundary, a horizontal center line that divides the second coding block equally and is parallel to the first upper boundary, and a vertical center line that divides the second coding block equally and is parallel to the first right boundary, when a second upper boundary of the first sub-coding block is adjacent to the horizontal center line but a second right boundary of the first sub-coding block is not adjacent to the first right boundary and the vertical center line, and the prediction direction is the upper right, the prediction circuit uses the reconstructed pixels of the adjacent block to generate the intermediate prediction mode. The video encoder of claim 4, wherein the second coding block includes an upper boundary, a first lower boundary, a left boundary, a right boundary, and a horizontal center line that divides the second coding block equally and is parallel to the upper boundary, and when the first sub-coding block is adjacent to the left boundary but a second lower boundary of the first sub-coding block is not adjacent to the first lower boundary and the horizontal center line, and the prediction direction is the lower left, the prediction circuit uses the reconstructed pixels of the adjacent block to generate the intermediate prediction mode. A video encoder as claimed in claim 1, wherein a prediction direction of the basic prediction mode is one of an upper direction, an upper right direction, a left direction and a lower left direction. A video encoding method, comprising: performing a first optimization operation on a first sub-coding block to select one of original pixels or reconstructed pixels of an adjacent block of the first sub-coding block according to a basic prediction mode to generate an intermediate prediction mode of the first sub-coding block, wherein when the reconstructed pixels of the adjacent block are not generated, the original pixels of the adjacent block are selected to generate the intermediate prediction mode; performing a second optimization operation on the first sub-coding block to select one of original pixels or reconstructed pixels of an adjacent block of the first sub-coding block according to a basic prediction mode to generate an intermediate prediction mode of the first sub-coding block; The first sub-coding block is used to generate a prediction mode based on the original pixels of the first sub-coding block, the reconstructed pixels of the adjacent block of the first sub-coding block, and the prediction mode; the prediction information of the first sub-coding block is generated according to the original pixels of the first sub-coding block, the reconstructed pixels of the adjacent block of the first sub-coding block, and the prediction mode, and the prediction information includes the prediction pixel and the residual value between the original pixel and the prediction pixel; the first sub-coding block is used to generate a coding coefficient and the reconstructed pixel according to the prediction information of the first sub-coding block; and a bit stream is generated according to the coding coefficient and the prediction information. The method of claim 10 further comprises: performing the second optimization operation on the first sub-coding block and performing the first optimization operation on a second sub-coding block at the same time, the first sub-coding block and the second sub-coding block are continuous in coding order; wherein, in a prediction direction of the basic prediction mode, the first sub-coding block is an adjacent block of the second sub-coding block, and the first optimization operation is performed on the second sub-coding block according to the original pixels of the first sub-coding block. As in the method of claim 10, the video coding method sequentially processes a first coding block, a second coding block and a third coding block arranged continuously in a frame, the second coding block is located between the first coding block and the third coding block, the second coding block includes a plurality of sub-coding blocks, the first sub-coding block is one of the plurality of sub-coding blocks, and the first optimization operation includes the following steps: According to a prediction direction and a position of the first sub-coding block relative to the second coding block, determine to use the original pixels or reconstructed pixels of the adjacent block of the first sub-coding block to generate the intermediate prediction mode.