WO2023101663A1

WO2023101663A1 - Method and apparatus of power control based on block activities

Info

Publication number: WO2023101663A1
Application number: PCT/US2021/061300
Authority: WO
Inventors: Yutian Feng; Yujie Hu; Steven SERTILLANGE
Original assignee: Picoai Ltd
Current assignee: Picoai Ltd
Priority date: 2021-12-01
Filing date: 2021-12-01
Publication date: 2023-06-08
Anticipated expiration: 2024-06-01

Abstract

A method for dynamic power control applied to integrated circuits is disclosed. According to this method, two or more hardware units in the integrated circuits are identified. Block activities for each of said two or more hardware units are determined. The total power is estimated based on the block activities associated with said two or more hardware units. Dynamic power controls are determined based on estimated total power. The dynamic power controls are then provided to said two or more hardware units to avoid possibility of chip overheat due to high power consumption.

Description

Method and Apparatus of Power Control Based on Block Activities

FIELD OF THE INVENTION

[0001] The present invention relates to integrated circuits design. In particular, the present invention relates to dynamic power control for integrated circuits such as application specific integrated circuits (ASIC).

BACKGROUND AND RELATED ART

[0002] The power dissipated on-chip can increase temperatures beyond recommended operating limits and can be a major concern for integrated circuit design. In order to avoid high temperature, the chip may have to be operated with reduced operating frequencies of clock or operating voltages, which will affect the chip performance. Therefore, the power for integrated circuits has to be properly managed to achieve a desired performance goal. For example, the power control can be designed to optimize the chip performance while avoiding over-heating. In another example, the power control can be designed to minimize the power consumption while maintaining a pre-specified performance.

[0003] The power consumed by integrated circuits fluctuates according to the activities of the integrated circuits. For example, the integrated circuits may be configured to perform different tasks such as matrix-vector multiplications associated with different vector dimensions. For a vector with a larger dimension, the circuits have to perform more logic operations (i. e. , more activities) to complete the task. On the other hand, a vector with a smaller dimension, the circuits need to perform less logic operations (i. e. , less activities) to complete the task. Furthermore, during executing a task, the circuits may be in different phases of the task (e.g., loading data, performing multiplication, etc.). The circuit activities may be higher in one instance and lower in another instance. Therefore, the power consumption incurred by the integrated circuits is dynamic in nature. Dynamic power refers to the power dissipated in the integrated circuits when the inputs switch from one level to another. Dynamic power is the major component of the power dissipated in circuits and also contributes to the peak power. It is a function of the supply voltage and the switching frequency (or clock rate).

[0004] Dynamic power control is a very important technology for a product and system. For an ASIC (application specific integrated circuits) product, the popular dynamic power control methods are:

• Limit power when temperature is high.

• Limit power when power supply detects power consumption is over the limit.

[0005] It’s easy to see both the above two methods are slow to react. In particular, when high temperature is detected, it may be too late to react. There may be milliseconds to seconds delay from detecting power over the limit to take effect to limit the power. The system may fail even encountering a very short period of instability caused by the power peaks. In order to guarantee the stability of the system, a significant margin is often added in thermal design and power supply redundancy. This may increase the cost and size of the system or unnecessarily lower the system performance.

[0006] Accordingly, it is desirable to design a fast reaction power control method to measure and manage power in hardware logic.

BRIEF SUMMARY OF THE PRESENT INVENTION

[0007] A method for dynamic power control applied to integrated circuits is disclosed. According to this method, two or more hardware units in the integrated circuits are identified. Block activities for each of said two or more hardware units are determined. The total power is estimated based on the block activities associated with said two or more hardware units. Dynamic power controls are determined based on estimated total power. The dynamic power controls are then provided to said two or more hardware units to avoid possibility of chip overheat due to high power consumption.

[0008] In one embodiment, the block activities for each of said two or more hardware units corresponds to a logic operation active level or a number of circuit switchings associated with each of said two or more hardware units.

[0009] In one embodiment, the block power consumed by each of said two or more hardware units is estimated by multiplying the block activities with a power factor for each of said two or more hardware units. The power factor for each of said two or more hardware units can be extracted by characterizing each of said two or more hardware units through power simulation in each functional mode. The functional mode is directly linked to the control command of the hardware unit. Thus, the relationship is built between the unit’s control command and power value.

[0010] In one embodiment, the total power estimated for said two or more hardware units is determined at intervals, wherein each interval corresponds to N clock cycles and N is a positive integer. The dynamic power controls can be determined according to the estimated total power calculated for a window period corresponding to M intervals, and wherein M is a positive integer. Furthermore, the dynamic power controls can be determined according to accumulated estimated total power calculated for the window period.

[0011] In one embodiment, a FIFO buffer, an adder and a register can be utilized to calculate the accumulated estimated total power calculated for the window period, and wherein the FIFO buffer is used to store the estimated total power calculated for the window period, the register is used to store a current accumulated estimated total power, and the adder is used to provide a next accumulated estimated total power by adding a current estimated total power and the current accumulated estimated total power and subtracting a stored estimated total power at M intervals before.

[0012] In one embodiment, the dynamic power controls are determined by comparing the accumulated estimated total power with one or more thresholds. In one embodiment, said one or more thresholds comprise a first threshold corresponding to a high accumulated total power level and the dynamic power controls comprise two or more power throttle settings, and wherein a most aggressive power throttle control setting among said two or more power throttle settings is selected when the accumulated estimated total power is greater than first threshold. In another embodiment, said one or more thresholds comprise a first threshold corresponding to a high accumulated total power level and a second threshold corresponding to a low accumulated total power level, and the dynamic power controls comprise three power throttle settings, and wherein a most aggressive power throttle setting among the three power throttle settings is selected when the accumulated estimated total power is greater than the first threshold, a middle aggressive power throttle setting among the three power throttle settings is selected when the accumulated estimated total power is greater than the first threshold when the accumulated estimated total power is smaller than the first threshold and greater than the second threshold, and a least aggressive power throttle setting among the three power throttle settings is selected when the accumulated estimated total power is smaller than the second threshold.

[0013] In one embodiment, the dynamic power controls comprise adjusting operating speed of a selected hardware unit.

[0014] A computer-implemented method for facilitating dynamic power control applied to integrated circuits is also disclosed.

BRIEF DESCRIPTION OF THE FIGURES

[0015] Fig. 1 illustrates an exemplary block diagram for the moving window based dynamic power control.

[0016] Fig. 2 illustrates an example of how the power is limited in average to avoid being over a high power threshold.

[0017] Fig. 3 illustrates a popular system level structure for power solution, where the ASIC chip incorporates the dynamic power throttle control according to the present invention.

[0018] Fig. 4 illustrates an exemplary comparison between a conventional power throttle control and the moving window power throttle control according to the present invention.

[0019] Fig. 5 illustrates an exemplary flowchart for dynamic power control applied to integrated circuits according to an embodiment of the present invention.

DETAILED DESCRIPTION

[0020] The following description is presented to enable any person skilled in the art to make and use the embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the embodiments described herein are not limited to the embodiments shown, but are to be accorded the widest scope consistent with the principles and features disclosed herein.

[0021] As mentioned before, according one conventional dynamic power control strategy, the chip temperature or system temperature is monitored or measured. When high temperature is detected, dynamic power control is engaged to slow down or pause the system operations. According another conventional dynamic power control strategy, the power consumption is measured or monitored. When the power consumption exceeds a limit, dynamic power control is engaged to slow down or pause the system operations. Either approach is slow to react and causes cost increase. In order to overcome the issues, the present invention discloses a dynamic power control scheme based on activities of the integrated circuits. In particular, the activities are measured for blocks of the integrated circuits. A block of the integrated circuits may correspond to a functional block. For example, a block may represent a group of hardware circuits that are used to implement an independent function controlled by a command such as a MAC (multiplication and accumulation) block or an activation function block for an artificial intelligence (Al) processor. However, other partition schemes may also be applied to divide the integrated circuits into blocks.

[0022] In order to estimate the power consumption of a block or a hardware unit in integrated circuits, embodiments of the present invention estimates power consumption from the activities of each block or each hardware unit. The activity associated with a block may correspond to the logic operation active level or the number of circuit switchings associated with the block. The power consumption of the block is then estimated according to the block activities. The activity measurement of blocks z, z=l, ... , n can be represented by act_level-z. Since activity levels from blocks may be in different units, they are multiplied by Pwr factorl-z to obtain a normalized power estimation value (e.g., power value per cycle). Pwr factor-z for block i can be extracted by characterizing the designs through power simulation in each functional mode. The functional mode is directly linked to the control command of the block. Thus, the relationship is built between block’s control command and power value. They can also be fine-tuned based on the actual hardware.

[0023] According to embodiments of the present invention, the block activities are measured once for every N (a positive integer) cycles, denoted as Pre_accum_rate in this disclosure. Furthermore, in order to ensure the reliability of the dynamic power control, a moving window based power estimation is applied. The sliding window size is defined as a multiple (M, a positive integer) of N cycles, where M is denoted as Win_size in this disclosure. Accordingly, the total delay corresponds to Win size * Pre accum rate cycles. In the moving-window power throttle block, the normalized power estimation value is preaccumulated and delayed (e.g., using a FIFO SRAM). Pre-accumulation is controlled by the parameter Pre accum rate. The delay is controlled by the parameter Win size. The purpose of pre-accumulation is to reduce FIFO SRAM size when a large window is needed. The power control occurs at every Pre accum rate cycles instead of every cycle.

[0024] For moving window based power control, the pre-accumulated power value Pre_accum_pwr is defined. The total power of the past cycles in the window of

Win_size*Pre_accum_rate can be accumulated according to:

Power accum win’ = Power accum win + Pre accum_pwr before delay -

Pre_accum_pwr after delay. (1)

[0025] In the above equation, Power accum win is the updated accumulated total power of the past cycles for the next window.

[0026] According to embodiments of the present invention, the accumulated total power of the past cycles is used to derive the dynamic power control. For example, Power accum win can be compared with one or more selected thresholds (e.g., Hi_pwr_threshold and Lo_pwr_threshold). Based on the comparison result, different power settings (e.g., Hi/Mid/Lo_pwr_idle_sets) can be selected to limit activity by providing power control signals (e.g., block_ctrll, block_ctrl2, ... ) to the circuit blocks. There will be some delay for the block to react after receiving the block control signal. Limiting activity can be, for example, throttling the rate of control signals sent to block funci, block_func2, ... , block funcn, which in turn slows down the execution of the function of a corresponding block.

[0027] The dynamic power control settings according to the present invention are based on the total power. The total power can be the accumulated total power associated with the moving window such as Power accum win as defined above. For example, the selection rule for the dynamic power control setting can be based on:

Hi_pwr_idle_sets : Power_accum_win >= Hi_pwr_threshold

Mid_pwr_idle_sets : Hi_pwr_threshold > Power_accum_win >= Lo_pwr_threshold

Lo_pwr_idle_sets : Power_accum_win < Lo_pwr_threshold

[0028] Fig. 1 illustrates an exemplary block diagram for the moving window based dynamic power control. In the Fig. 1, act level-/ (110-1, ... , 110-n) is the activity measurement of blocks z, z=l,... ,n, and multipliers (112-1,... ,112-n) are used to implement the multiplication of act level-z and Pwr_factorl-z. The estimated block powers are summed up using adder 114 to obtain the estimated pre-accumulated total power. The estimated preaccumulated total power is measured once every Pre accum rate cycles as implemented by Pre accum unit 130. For convenience, the unit of Pre accum rate cycles is also referred as one window slot or one interval. The delay parameter Win_size can be programmed to store at delay unit 140, which is used to control the FIFO SRAM 150. According to Win size, the FIFO SRAM will output a Pre_accum_pwr that is measured at Win size window slots before. The adder 160 in Fig. 1 performs the operations corresponding to eqn. (1) where the three inputs correspond to the current Power accum win, the current Pre_accum_pwr (i.e., Pre_accum_pwr before delay) and Pre_accum_pwr measured at Win size window slots before (i.e., Pre_accum_pwr after delay). The sign associated with Pre_accum_pwr after delay indicates subtraction. The output from the adder 160 corresponds to the updated Power accum win, which becomes the current Power accum win in the next window slot. Effectively, the arrangement associated with the FIFO SRAM 150, Adder 160 and register or delay 162 computes the accumulated total power over a moving window with Win size window slots.

[0029] The derived Power accum win is then compared with thresholds Hi_pwr_threshold and Lo_pwr_threshold using compare unit 170. The comparison result is then used to select (using select unit 172) a power control setting among Hi_pwr_idle_sets, Mid_pwr_idle_sets and Lo_pwr_idle_sets. The selected power control setting is used to generate control signals (180-1, 180-2, ... ) for individual blocks.

[0030] Hi_pwr_threshold is a key parameter in the dynamic power control design. It specifies the highest power value the system can handle. When the power estimation value is over this threshold, the parameter Hi_pwr_idle_sets will be selected as control signals block ctrll, block_ctrl2, ... block ctrln to reduce the activity of corresponding blocks. For example, the operating rate can be reduced by techniques such as inserting idle cycles on the control path of the blocks, which in turn inserts idle cycles during the function execution.

[0031] There is a delay from the moment of control signal block ctrlz (z=l , ... , n) being issued to block funcz to react when Power accum win is over Hi_pwr_threshold. Therefore, the power will continue to increase for a short period of time. If Hi_pwr_threshold is the only threshold used to control dynamic power, the power may go up too rapidly and overshoot the budget until the control block throttles the activities. To avoid it, Lo_pwr_threshold and Mid_pwr_idle_sets are also included to prevent the power value to increase too fast when the power is close to Hi_pwr_threshold.

[0032] Accordingly, the setting rule becomes:

Hi_pwr_idle_sets: the most aggressive settings to throttle blocks’ activities

Mid_pwr_idle_sets : mild setting to throttle blocks’ activities

Lo_pwr_idle_sets : settings not or little to throttle blocks’ activities

Hi_pwr_threshold: its value needs to be set at the power budget. When the required function’s power at full speed is more than this threshold, the power will be limited to a level around this threshold.

Lo_pwr_threshold: its value is based on how fast the power increases when the system runs in full speed. It can be close to Hi_pwr_threshold if it is not too fast and vice versa.

[0033] For the best performance, Lo_pwr_idle_sets can be set to make the system run in full speed. If power target is much lower than the power when hardware is running in full speed, Lo_pwr_idle_sets can be set to limit block activities in some small rate. It imposes the power limitation at the beginning that will limit the power increase rate to some degree and help to improve system stability.

[0034] For a system which is very sensitive to power peaks, more thresholds and power idle settings can be implemented to make the power curve flatter during high power tasks.

[0035] The selection of the window size (i.e. , Win_size*Pre_accum_rate) is also a very important design parameter. The bigger the window size is selected, the slower the power control loop is. It needs to be related to the RC characteristics of the ASIC power rail, IO (input and output) and caps (capacitors). The ASIC power rail and small caps on board can absorb small power jitter. The thermal and power supply surge detection are very slow. Thus, the window size needs to be set in the middle to patch the gap. Because Win size is limited by FIFO SRAM depth, Pre accum rate can be increased for bigger window sizes.

[0036] Fig. 2 illustrates an example of how the power is limited in average to avoid being over Hi_pwr_threshold according to an embodiment of the present invention. The dash-dot line 210 represents the power when the system runs at full speed in some high activity tasks.

Thus, the system runs in the following steps:

1) At the beginning, the system runs un-throttled with average activity. Power_accum_win is accumulating and reaches the dash-dot line (point 1 in Fig. 2).

2) The system switches to high power tasks. Then we can see Power_accum_win increases very quickly. After it crosses over Lo_pwr_threshold (point 2 in Fig. 2), Mid_pwr_idle_sets is used to limit the power value increasing rate. The power increase starts to slow down after a delay (point 3 in Fig. 2). But the power under Mid_pwr_idle_sets is still sufficient to limit the power. Therefore, Power_accum_win continues increasing until it crosses over Hi_pwr_threshold (point 4 in Fig. 2).

3) When Power_accum_win is over Hi_pwr_threshold, Hi_pwr_idle_sets takes effect. After a delay, the power starts to drop (point 5 in Fig. 2).

4) Power_accum_win is decreasing until it is lower than Hi_pwr_threshold (point 6 in Fig. 2).

5) During the high power task period, steps 3 and 4 occur repeatedly until the high power task finishes.

[0037] The power curve under the moving window dynamic power control in the example of Fig. 2 is indicated by line segments 220. The periods that Mid_pwr_idle_sets is selected are indicated by thick line segments 230. The periods that Hi_pwr_idle_sets is selected are indicated by thick line segments 232. The period that Lo_pwr_idle_sets is selected is indicated by interval 234.

[0038] The moving window power throttle architecture according to the present invention relies on the power estimation value from block activity levels. While the estimated power may not be very accurate, the inaccuracy can be corrected in the system level power solution. Fig. 3 illustrates a popular system level structure for power solution, where the ASIC chip incorporates the dynamic power throttle control according to the present invention. The system comprises an ASIC chip 350, an external power source 310, DC-DC converter 320, MCU 330 and temperature sensors 340. The ASIC chip comprises CPU 352, moving_win_power_throttle unit 354 and various functional blocks 356. The power measurement in DC-DC converter is accurate. Temperature sensors 340 will report the temperatures in various locations on the chip. Based on the information, MCU 330 on board and/or CPU 352 on chip can be configured to perform global power control. Because the global power control is accessed using a slow interface, for example I2C, the reaction is on the milliseconds to seconds order, which is too slow for accurately controlling the power consumption.

[0039] Fig. 4 illustrates an exemplary comparison between a conventional power throttle control and the moving window power throttle control according to an embodiment of the present invention. In this case, even if the power estimation based on block activities is rough, it can still do a good job to limit the peak power in short periods of time. Under the control of moving window power throttle, the system power could be a little higher or lower than the power budget. However, after the correction by global power control, the average power in extended period could be very close to the power budget.

[0040] If the power is managed by the global control only, the piece-wise line segments 410 correspond to the possible waveform in the Fig. 4. There may be some moments with exceedingly high power peaks in short period (e.g., period 430). After the high power peak is detected, some aggressive action needs be adopted to limit the power. Then there may be some over-constrained periods (e.g., periods 432 and 434) that cause lower performance. Besides, to make the system stable during the high power peak period, the power supply solution on board and on chip needs to be enhanced. It is much more expensive than the power throttle logic of the present invention. In Fig. 4, the zig-zag waveform 420 represents the power consumption for the ASIC incorporating moving window dynamic power control according to the present invention, where the power consumption fluctuates around the target power budget with small deviations. [0041] Fig. 5 illustrates an exemplary flowchart for dynamic power control applied to integrated circuits according to an embodiment of the present invention. According to this method, two or more hardware units in the integrated circuits are identified in step 510. Block activities for each of said two or more hardware units are determined in step 520. The total power is estimated based on the block activities associated with said two or more hardware units in step 530. Dynamic power controls are determined based on estimated total power in step 540. The dynamic power controls are then provided to said two or more hardware units to avoid possibility of chip overheat due to high power consumption in step 550.

[0042] The methods and processes described above can be embodied as code and/or data, which can be stored in a computer-readable storage medium as described above. When a computer system or a digital processing system reads and executes the code and/or data stored on the computer-readable storage medium, the computer system or the digital processing system performs the methods and processes embodied as data structures and code and stored within the computer-readable storage medium. For example, the cache tag control block can be implemented by executing codes, micro-codes, or instructions on a CPU, a digital processor, a micro-controller, a programmable device, etc. Upon executing codes, micro-codes, or instructions, one or more cache operation commands are generated to operate the data cache.

[0043] Furthermore, the methods and processes described above can be included in hardware modules. For example, the hardware modules can include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), and other programmable-logic devices now known or later developed. When the hardware modules are activated, the hardware modules perform the methods and processes included within the hardware modules.

[0044] The foregoing embodiments described herein have been presented for purposes of illustration and description only. They are not intended to be exhaustive or to limit the embodiments described herein to the forms disclosed. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. Additionally, the above disclosure is not intended to limit the embodiments described herein. The scope of the embodiments described herein is defined by the appended claims.

Claims

1. A method for dynamic power control applied to integrated circuits, wherein the method comprises: identifying two or more hardware units in the integrated circuits; determining block activities for each of said two or more hardware units; estimating total power based on the block activities associated with said two or more hardware units; determining dynamic power controls based on estimated total power; and providing the dynamic power controls to said two or more hardware units to avoid possibility of chip overheat due to high power consumption.

2. The method of Claim 1, wherein the block activities for each of said two or more hardware units corresponds to a logic operation active level or a number of circuit switchings associated with each of said two or more hardware units.

3. The method of Claim 1, wherein block power consumed by each of said two or more hardware units is estimated by multiplying the block activities with a power factor for each of said two or more hardware units.

4. The method of Claim 3, wherein the power factor for each of said two or more hardware units is extracted by characterizing each of said two or more hardware units through power simulation in each of functional modes of each of said two or more hardware units.

5. The method of Claim 1, wherein the total power estimated for said two or more hardware units is determined at intervals, wherein each interval corresponds to N clock cycles and N is a positive integer.

6. The method of Claim 5, wherein the dynamic power controls are determined according to the estimated total power calculated for a window period corresponding to M intervals, and wherein M is a positive integer.

7. The method of Claim 6, wherein the dynamic power controls are determined according to accumulated estimated total power calculated for the window period.

8. The method of Claim 7, wherein a FIFO buffer, an adder and a register are utilized to calculate the accumulated estimated total power calculated for the window period, and wherein the FIFO buffer is used to store the estimated total power calculated for the window period, the register is used to store a current accumulated estimated total power, and the adder is used to provide a next accumulated estimated total power by adding a current estimated total power and the current accumulated estimated total power and subtracting a stored estimated total power at M intervals before.

9. The method of Claim 7, wherein the dynamic power controls are determined by comparing the accumulated estimated total power with one or more thresholds.

10. The method of Claim 9, wherein said one or more thresholds comprise a first threshold corresponding to a high accumulated total power level and the dynamic power controls comprise two or more power throttle settings, and wherein a most aggressive power throttle control setting among said two or more power throttle settings is selected when the accumulated estimated total power is greater than first threshold.

11. The method of Claim 9, wherein said one or more thresholds comprise a first threshold corresponding to a high accumulated total power level and a second threshold corresponding to a low accumulated total power level, and the dynamic power controls comprise three power throttle settings, and wherein a most aggressive power throttle setting among the three power throttle settings is selected when the accumulated estimated total power is greater than the first threshold, a middle aggressive power throttle setting among the three power throttle settings is selected when the accumulated estimated total power is greater than the first threshold when the accumulated estimated total power is smaller than the first threshold and greater than the second threshold, and a least aggressive power throttle setting among the three power throttle settings is selected when the accumulated estimated total power is smaller than the second threshold.

12. The method of Claim 1, wherein the dynamic power controls comprise adjusting operating speed of a selected hardware unit.

13. A computer-implemented method for facilitating dynamic power control applied to integrated circuits, the method comprising: identifying two or more hardware units in the integrated circuits; determining block activities for each of said two or more hardware units; estimating total power based on the block activities associated with said two or more hardware units; determining dynamic power controls based on estimated total power; and providing the dynamic power controls to said two or more hardware units to avoid possibility of chip overheat due to high power consumption.

16