CN114692810A - Device and board card for calculating Winograd convolution - Google Patents

Abstract

The present invention relates to a device and a board for calculating Winograd convolution, wherein the device is connected to an off-chip memory, and the off-chip memory stores neuron data and a plurality of instructions. The apparatus includes a neuron buffer and a forward transformation unit. The neuron buffer is used to start loading the neuron data from the off-chip memory; and the forward transformation unit is used to start reading the neuron data from the neuron buffer before the neuron data is loaded. The metadata is forward transformed to generate forward transformed data. The invention has the technical effects of ensuring network precision, performance acceleration, area reduction and power consumption reduction.

Description

Devices and boards for computing Winograd convolutions

technical field

The present invention generally relates to the field of neural networks. More specifically, the present invention relates to devices and boards for computing Winograd convolutions.

Background technique

With the rapid development of the information age, research in the field of artificial intelligence and machine learning is hot, and related industries are booming. Convolutional neural networks have a wide range of roles in computer vision, autonomous driving, machine translation, speech recognition, smart homes, and more.

The convolutional neural network has a large amount of parameters and a large amount of calculation, which makes the convolutional neural network model seriously limit its execution performance under the limited area and computing power of the portable mobile terminal. It will also cause a huge cost of power consumption.

Winograd convolution is a convolution acceleration implementation based on polynomial interpolation algorithm. It divides the two inputs of the convolution operation: neurons and weights to a certain scale, and then performs linear transformation, that is, Winograd forward transformation, and then multiplies the transformed neurons and weights. Perform a linear transformation on the bit multiplication result again, that is, Winograd inverse transformation, and finally obtain a convolution result equivalent to the original convolution operation.

Since in the process of Winograd convolution operation, the forward and inverse transformation matrices of neurons and weights are composed of simple fixed values, so the forward and inverse transformation process of Winograd neurons and weights can be realized only by addition. However, the multiplication operation required in the Winograd algorithm only occurs in the process of bitwise multiplication, and the multiplication complexity of this process is considerably reduced compared to the original convolution algorithm. Since the overhead (timing, power consumption, area) of the multiplication operation in hardware is much higher than the addition of the same bit width, replacing the original convolution operation with Winograd convolution can bring obvious benefits in hardware energy efficiency ratio and operation time.

However, there is currently no hardware designed for the Winograd convolution acceleration algorithm, so that the existing artificial intelligence chips cannot fully display the advantages of the Winograd convolution operation. Therefore, a hardware device that can efficiently run the Winograd convolution algorithm is urgently needed.

SUMMARY OF THE INVENTION

In order to at least partially solve the technical problems mentioned in the background art, the solution of the present invention provides a device and a board for calculating Winograd convolution.

In one aspect, the present invention discloses an apparatus for computing Winograd convolutions, connected to off-chip memory, where neuron data and a plurality of instructions are stored in the off-chip memory. The device includes: a neuron cache for enabling loading of the neuron data from the off-chip memory; and a forward transformation unit for enabling the neuron data to be loaded from the neuron before the neuron data is loaded The buffer reads the neuron data and performs forward transformation to generate forward transformation data.

In another aspect, the present invention discloses an integrated circuit device including the aforementioned device, and a board including the aforementioned integrated circuit device.

The hardware structure proposed by the invention can match the Winograd convolution acceleration algorithm, and has the technical effects of ensuring network accuracy, performance acceleration, area reduction and power consumption reduction.

Description of drawings

The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown by way of example and not limitation, and like or corresponding reference numerals refer to like or corresponding parts wherein:

FIG. 1 is a schematic diagram illustrating the conversion of the original convolution of F (2×2, 3×3) into a Winograd convolution;

2 is a structural diagram illustrating a board according to an embodiment of the present invention;

3 is a structural diagram illustrating an integrated circuit device according to an embodiment of the present invention;

4 is a schematic diagram illustrating an internal structure of a computing device according to an embodiment of the present invention;

5 is a schematic diagram illustrating a forward transformation data cache according to an embodiment of the present invention;

6 is a schematic diagram illustrating a bitwise multiply-accumulate operator according to an embodiment of the present invention;

FIG. 7 is a schematic diagram illustrating a pipeline of an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

It should be understood that the terms "first", "second", "third" and "fourth" in the claims, description and drawings of the present invention are used to distinguish different objects, rather than to describe a specific order . The terms "comprising" and "comprising" used in the description and claims of the present invention indicate the presence of the described features, integers, steps, operations, elements and/or components, but do not exclude one or more other features, integers , step, operation, element, component and/or the presence or addition of a collection thereof.

It should also be understood that the terminology used in this specification of the present invention is for the purpose of describing particular embodiments only, and is not intended to limit the present invention. As used in the present specification and claims, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise. It will be further understood that, as used in the present specification and claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items.

As used in this specification and in the claims, the term "if" may be contextually interpreted as "when" or "once" or "in response to determining" or "in response to detecting".

The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Winograd convolution acceleration algorithm (hereinafter referred to as Winograd algorithm or Winograd convolution) is to use the linear transformation of the operands in the convolution operation, and then find the transformation method that requires the smallest number of multiplications, and then replace the required multiplication by adding some addition operations. operate. From the perspective of hardware, multipliers have more complex structure, larger area power consumption, and worse comprehensive processing performance than adders. Therefore, the Winograd algorithm, which replaces multiplication with addition, has great advantages in processing two-dimensional convolution operations. big advantage.

For two-dimensional convolution, the convolution result can be expressed as F(m×n, r×s), that is, the output shape is m×n, and the weight shape is r×s. The matrix representation of the Winograd algorithm is:

Y=A ^T [(GgG ^T )⊙(B ^T dB)]A

Among them, Y represents the output matrix of the convolution operation, A ^T is the inverse transformation left multiplication constant matrix, G is the weight transformation left multiplication constant matrix, g is the weight of the original convolution, G ^T is the weight transformation right multiplication constant matrix , ⊙ means bitwise multiplication, B ^T is the neuron transformation left multiplication constant matrix, d is the neuron data, B is the neuron transformation right multiplication constant matrix, and A is the inverse transformation right multiplication constant matrix. The left-multiplied and right-multiplied matrices of each transformation are only transposed.

Taking F(2×2,3×3) as an example, the aforementioned constant matrices are as follows:

Figure 1 shows a schematic diagram of the conversion of the original convolution of F (2×2, 3×3) into a Winograd convolution. As shown in the figure, the neuron data 101 and the convolution kernel 102 are subjected to a convolution operation. During calculation, the neuron data 101 is arranged in a row according to the elements in the sliding window 103, the sliding window 103 is slid 4 times to form a 4×9 matrix 104, and then the elements of the convolution kernel 102 are arranged in a column to form a 9×1 matrix 105 , a 4×9 matrix 104 and a 9×1 matrix 105 perform a convolution operation to obtain a 4×1 convolution result 106 .

Then according to the dotted line in the figure, the 4×9 matrix 104 is transformed into a 2×3 matrix 107, the 9×1 matrix 105 is transformed into a 3×1 matrix 108, and the 4×1 convolution result 106 is transformed into a 2×1 convolution result. 109. After the linear transformation, the first element of the 2x1 convolution result 109 is R ₀ =M ₀ +M ₁ +M ₂ , and R ₁ =M ₁ -M ₂ -M ₃ . And M ₀ , M ₁ , M ₂ , and M ₃ can be represented by the following formulas:

Through the aforementioned segmentation and linear transformation, the original convolution operation needs to perform 36 multiplications, while the Winograd algorithm only needs to perform 16 multiplications, reducing the multiplication complexity by 2.25 times.

It can be seen from the conversion of the above two-dimensional convolution Winograd algorithm that the Winograd algorithm is mainly divided into the following steps. First, perform the left and right multiplication of the weight constant matrix on the weights, namely GgG ^T , to obtain the weights after the linear transformation of Winograd; at the same time, perform the left and right multiplication of the neuron constant matrix on the neuron data, namely B ^T dB, get the neuron after the Winograd linear transformation. Furthermore, perform the multiplication operation of the neuron and the weight matrix after the Winograd transformation, namely (GgG ^T )⊙(B ^T dB), to obtain the result of the multiplication of the opposite positions. Finally, the left multiplication and right multiplication of the Winograd inverse transformation constant matrix will be performed on the bit multiplication result, that is, A ^T [(GgG ^T )⊙(B ^T dB)]A, and finally the convolution result equivalent to the original convolution is obtained. .

From the perspective of hardware design, the present invention implements a pipeline design for these three large transformation steps to achieve more efficient acceleration performance in view of the dependencies among the above three processes and the distinguishing features of operations.

FIG. 2 shows a schematic structural diagram of a board 20 according to an embodiment of the present invention. As shown in FIG. 2 , the board 20 includes a chip 201, which is a system-on-chip (SoC), or a system-on-a-chip, and integrates one or more combined processing devices. The combined processing device is an artificial The intelligent computing unit is used to support various deep learning and machine learning algorithms to meet the intelligent processing requirements in complex scenarios in the fields of computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud intelligence applications is the large amount of input data, which has high requirements on the storage capacity and computing power of the platform. The board 20 in this embodiment is suitable for cloud intelligence. applications, with huge off-chip storage, on-chip storage and massive computing power.

The chip 201 is connected to an external device 203 through an external interface device 202 . The external device 203 is, for example, a server, a computer, a camera, a monitor, a mouse, a keyboard, a network card or a wifi interface, and the like. The data to be processed can be transmitted to the chip 201 by the external device 203 through the external interface device 202 . The calculation result of the chip 201 can be transmitted back to the external device 203 via the external interface device 202 . According to different application scenarios, the external interface device 202 may have different interface forms, such as a PCIe interface and the like.

The board 20 also includes a storage device 204 for storing data, which includes one or more storage units 205 . The storage device 204 is connected to the control device 206 and the chip 201 through a bus and performs data transmission. The control device 206 in the board 20 is configured to control the state of the chip 201 . To this end, in an application scenario, the control device 206 may include a microcontroller (Micro Controller Unit, MCU).

FIG. 3 is a block diagram showing the combined processing device in the chip 201 of this embodiment. As shown in FIG. 3 , the combined processing device 30 includes a computing device 301 , an interface device 302 , a processing device 303 and a DRAM 304 .

The computing device 301 is configured to perform operations specified by the user, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations, especially Winograd convolution operations, which can be communicated with the interface device 302. The processing device 303 interacts to jointly complete the operation specified by the user.

The interface device 302 is used to transmit data and control instructions between the computing device 301 and the processing device 303 . For example, the computing device 301 may obtain input data from the processing device 303 via the interface device 302 and write the input data into the storage device on-chip of the computing device 301 . Further, the computing device 301 can obtain the control instruction from the processing device 303 via the interface device 302 and write it into the control cache on the computing device 301 . Alternatively or alternatively, the interface device 302 may also read the data in the storage device of the computing device 301 and transmit it to the processing device 303 .

The processing device 303, as a general processing device, performs basic control including but not limited to data transfer, starting and/or stopping the computing device 301, and the like. Depending on the implementation, the processing device 303 may be one or more types of central processing unit (CPU), graphics processing unit (GPU), or other general-purpose and/or special-purpose processors Processors, these processors include but are not limited to digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other Programming logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and the number thereof can be determined according to actual needs. As mentioned above, only for the computing device 301 of the present invention, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 301 and the processing device 303 are considered together, the two are considered to form a heterogeneous multi-core structure.

The DRAM 304 is used to store the data to be processed, which is an off-chip memory, and the size is usually 16G or larger. It is used to save the data of the computing device 301 and/or the processing device 303, especially the neurons that are to be subjected to the Winograd convolution operation. data and weights. In this embodiment, the processing device 303 has previously linearly transformed the weight of the original convolution into a Winograd weight GgG ^T and stored in the DRAM 304 .

FIG. 4 shows a block diagram of the computing device 301 . The computing device 301 includes a bus 401 , a direct memory access (DMA) module 402 , an instruction buffer (Iram) 407 , a decoding unit (IDU) 408 , a neuron buffer (Nram) 409 , and a forward transformation unit (NTU, neuron transformation unit) 410 , forward transform data buffer (WNram) 411, weight buffer (Wram) 412, bitwise multiply-accumulate operator (MAC) 413, bitwise multiply data buffer (WRram) 414, inverse transform unit (ITU) 415, result buffer ( Rram) 416 and a logic operation module (ALU, arithmetic logic unit) 417.

The bus 401 is a public communication trunk line for transmitting information between various devices, and is a transmission harness composed of wires. According to the type of information transmitted by the combined processing device 30, the bus 401 is a general term for a data bus, an address bus and a control bus, used to transmit data. , data addresses and instructions. The bus 401 serves as a communication channel between the DRAM 304 and the computing device 301 , which is specifically PCIe in this embodiment.

The DMA module 402 is used for copying data from one address space to another address space, usually for transferring data between an external memory (eg, the DRAM 304 ) and the internal buffer of the computing device 301 . When implementing DMA transmission, the processing device 201 transfers the bus control right to the DMA module 402 , and the DMA module 402 controls the bus 401 to carry out data transfer.

The DMA module 402 includes neuron direct memory access (NDMA) 403 , weight direct memory access (WDMA) 404 , instruction direct memory access (IDMA) 405 and result direct memory access (RDMA) 406 . NDMA 403 is used to input neuron data from DRAM 304 , WDMA 404 is used to input Winograd weights from DRAM 304 , IDMA 405 is used to input commands from DRAM 304 , and RDMA 406 is used to output calculation results to DRAM 304 . In other embodiments, NDMA 403, WDMA 404, IDMA 405, and RDMA 406 may be implemented by the same direct memory access.

The Iram 407 is used to temporarily store the instructions input by the IDMA 405 , and the IDU 408 fetches the instructions from the Iram 407 for decoding, and controls other units to operate according to the decoded instructions. The IDU 408 is the decoding and scheduling unit of the entire computing device 301, and is responsible for decoding the control instructions obtained from the DRAM 304, converting them into control signals to coordinate the operation of each module/unit on the chip, and also responsible for order preservation, de-dependency, and execution of the instructions. Branch prediction, exception handling, interrupt handling and many other tasks. The thin line arrows in the figure are control flow, and the thick line arrows are data flow.

The Nram 409 is used to temporarily store the neuron data sent by the NDMA 403 according to the decoded instruction, and the NTU 410 reads the neuron data from the Nram 409 according to the decoded instruction to perform forward transformation, that is, the operation of B ^T dB , to generate the forward transformation data, and the generated forward transformation data is temporarily stored in the WNram 411 .

NTU 410 includes input buffers, register files, adder banks, and output buffers.

When the NTU 410 receives an instruction to load neuron data from the Nram 409, the input buffer is used as a FIFO queue buffer for temporarily storing the neuron data. The stage of loading neuron data will continue until all data reception is completed. Convolution filters of different scales will be configured with fixed and independent buffer resource partitioning and input counts. The overall process is controlled by instructions sent by the IDU 408 .

According to the decoded instructions and the planned operation sequence, the register file retrieves the temporarily stored neuron data from the input cache and stores it at the specific address of the register file. The neuron data stored in the specific address of the register file becomes an addition operation. number. In this embodiment, since the pipeline time lengths of the three stages of input, operation, and output are equal, the phenomenon of cache hardware resource dependence occurs. In order to solve the problem of resource dependence, the register file is divided into ping storage units of the same size and Pong storage unit, the i-th addition operand and the positive transformation data generated after the calculation are temporarily stored in the ping storage unit, and the i+1-th addition operand and the i+1-th positive transformation data are temporarily stored in the pong storage unit, The i+2th addition operand and the i+2th positive transformation data are temporarily stored in the ping storage unit, covering the ith addition operand and the i+2th positive transformation data, and the register file is stored according to this rule.

The adder group sequentially reads the addition operands from the specific address of the register file according to the decoded instruction to perform the addition operation. In this embodiment, the number of adder groups is 2 groups to correspond to the addition operation scheduling direction, each group includes 16 adders to correspond to the vectorization direction, each adder is an FB32 adder, in the channel direction of the neuron data The addition operation in the positive transformation of the Winograd convolution is performed in a specific order. The specific order is to first calculate the addition of the left multiplication matrix B ^T of the Winograd convolution, and then calculate the addition of the right multiplication matrix B of the Winograd convolution, and finally generate the positive transformation. data, and then store the positive transformation data back into the register file. The operation order, register allocation, and operation time are related to the scale of the convolution filter, and are controlled by the IDU408 sending instructions. This operation stage generates data dependencies with the aforementioned stage of loading neuron data, and is executed in a pipelined manner, which is implemented by hardware through counting.

The output buffer is also a first-in-first-out queue buffer for temporarily storing forward-transformed data sequentially from the ping storage unit and the pong storage unit. This output stage needs to rely on the overall completion of the operation stage before the corresponding cached output can be performed.

The WNram 411 includes a plurality of cache units, and FIG. 5 shows a schematic diagram of an exemplary WNram 411. As shown in the figure, the WNram 411 includes four cache units: a first cache unit 501, a second cache unit 502, and a third cache unit 503 and fourth cache unit 504 . Forward transformation data from NTU 410 is routed to one or more of these cache units.

Returning to FIG. 4 , the Wram 412 temporarily stores the Winograd weights sent by the WDMA 404 according to the decoded instructions, and the MAC 413 reads the Winograd weights from the Wram 412 according to the decoded instructions, and reads the positive transformation from the WNram 411 data, perform the multiplication and accumulation operation on the positive transformation data and the Winograd weight, that is, perform the operation of [(GgG ^T )⊙(B ^T dB)] to generate the multiplication data, and temporarily store the multiplication data. to WRram 414.

In this embodiment, the MAC 413 includes 64 MAC operators, which are equally divided into 4 groups to perform operations in 4 different batches, and the 16 MAC operators in each group are independently distributed. The positive transformation data of WNram411 needs to be sent to these 64 MAC operators at the same time, so that it can perform the multiplication and accumulation operation with different Winograd weights. Therefore, the WNram 411 sends the forward transformation data by broadcasting or routing. Due to the large output load, in order to ensure the driving ability and timing, the forward transformation data of WNram 411 is broadcast or distributed through the N1 and N2 levels, and is first sent to 4 N1 nodes, and each N1 node broadcasts or distributes the route to 4 N2 nodes. , each N2 node rebroadcasts or distributes routes to 4 MAC operators.

FIG. 6 shows a schematic diagram of a set of MAC operators 601 . The MAC operator 601 first multiplies the bits, and then accumulates the resultant vectors in sequence. The logical function is equivalent to calculating the inner product of vectors or performing the operation of element values in matrix multiplication.

ITU 415 reads the bitwise multiplication data from the WRram 414 according to the decoded instruction, and inversely transforms the bitwise multiplication data, that is, performs the operation of A ^T [(GgG ^T )⊙(B ^T dB)]A to obtain the volume Product results, convolution results are temporarily stored in Rram 416.

ITU 415 includes input buffers, register files, adder banks, and output buffers.

When the ITU 415 receives an instruction to load the bitwise multiplication data from the WRram 414, the input buffer is used as a FIFO queue buffer for temporarily storing the bitwise multiplication data. The stage of loading the bitwise multiplication data will continue until all data reception is completed, and the convolution filters of different scales will be configured with fixed and independent buffer resource partitioning and input counting. The overall process is controlled by the IDU 408 sending instructions.

According to the decoded instruction, according to the fixed operation sequence, the register file fetches the temporarily stored pairwise multiplication data from the input cache and stores it to a specific address of the register file. operand. Similarly, in order to solve the problem of resource dependence, the register file has a ping storage unit and a pong storage unit of the same size. The i-th addition operand and the convolution result generated after the calculation are temporarily stored in the ping storage unit. The addition operand and the i+1th convolution result are temporarily stored in the pong storage unit, and the i+2th addition operand and the i+2th convolution result are temporarily stored in the ping storage unit, covering the ith addition operation. number and the i-th convolution result, and the register file is stored according to this rule.

The adder group sequentially reads the addition operands from the specific address of the register file according to the decoded instruction to perform the addition operation. Same as the adder group, the number of adder groups is 2 groups to correspond to the addition operation scheduling direction. Each group includes 16 adders to correspond to the vectorization direction. The direction performs the addition operation in the inverse transformation of the Winograd convolution in a specific order. This specific order is to first calculate the addition of the left multiplication matrix A ^T of the Winograd convolution, and then calculate the addition of the right multiplication matrix A of the Winograd convolution, and finally generate the volume The result is accumulated, and the convolution result is stored back in the register file. The operation order, register allocation, and operation time are related to the scale of the convolution filter, and are controlled by the IDU408 sending instructions. This operation stage and the aforementioned stage of loading the bit-multiplied data generate data dependencies, which are executed in a pipelined manner and implemented by hardware through counting.

The output buffer is also a first-in-first-out queue buffer for temporarily storing the convolution results from the ping memory unit and the pong memory unit in sequence. This output stage needs to rely on the overall completion of the operation stage before the corresponding cached output can be performed.

In addition to the Winograd convolution, the computing device 301 can also perform all operations related to the neural network. The ALU 417 is used to perform two tasks according to the decoded instructions: the first task is the operation of the convolution fusion operation, that is, the operation can be combined with The convolutional layer performs operations on the chip that do not require more data at one time. These operations include activation, biasing, direction part, and accumulation operations; the second task is non-convolutional operations. The operation result generated by ALU 417 is also temporarily stored in Rram416. The existence of the ALU 417 can ensure that various operations in the convolutional neural network can be completely implemented in the computing device 301, so that the computing device 301 has the versatility and integrity of a neural network.

According to the decoded instruction, the RDMA 406 fetches the convolution result from the Rram 416 and outputs it to the DRAM 304, thus completing the entire convolution operation. Similarly, the RDMA 406 can also fetch other operation results generated by the ALU 417 from the Rram 416 and output them to the DRAM 304 according to the decoded instruction.

Due to the huge data scale of the convolution operation, in order to reduce the start-up overhead of the instruction itself, this embodiment further utilizes instruction control to enable related modules/units to execute pipelines, thereby improving the utilization rate of hardware.

It can be seen from the above that the input timing and data scale of neuron data will affect the positive transformation process of neurons in Winograd convolution instructions, and the input timing and data scale of weight data will also affect the registration process of Winograd convolution instructions. The completion timing of the Winograd convolution inverse transform will affect the execution of the convolution result output instruction. Therefore, from the point of view of control, the order of instructions and the time point of execution are critical. Not only that, but in this embodiment, synchronization instructions need to be inserted between the instructions with dependencies to solve the problem between the input and output programs and the Winograd volume. The data dependence problem of product programs.

FIG. 7 shows a schematic diagram of the pipeline of this embodiment. The IDU 408 mainly controls the pipeline operations among Nram 409 , NTU 410 , Wram 412 , MAC 413 , ITU 415 and Rram 416 .

When performing the i-th convolution operation, the IDU 408 sends an instruction to control the _Nram 409 to start loading the neuron data i from the DRAM 304 at the time point T1, and the loading of the neuron data i will be completed at the time point _T2 . Before the neuron data i is loaded, at the time point T ₃ , the IDU 408 controls the NTU 410 to start reading the neuron data i from the Nram 409 to perform forward transformation according to the synchronization command, so as to generate the forward transformation data i. From the time point T3, while the _Nram 409 loads the neuron data i, the NTU 410 also reads the neuron data i from the Nram 409 to perform forward transformation, and the forward transformation of the data i will be completed at the time point T4 _.

The convolution operation of neuron data i needs to be matched with Winograd weight i. Based on the hardware structure of this embodiment, NDMA 403 is responsible for the input of neuron data i, and WDMA 404 is responsible for the input of Winograd weight i, which can be parallelized, but consider The input and output bandwidth to the computing device 301 is fixed, and the neuron data i needs to be forward transformed by the NTU 410 before it is multiplied and accumulated by the MAC 413 with the Winograd weight i. Therefore, in this embodiment, the neuron data is designed so that the i is _first loaded at time point T1 and forward transformed at time point T3, and Winograd weight i is input to _Wram 412 later, so that Nram409, NTU 410, Wram 412 and MAC 413 can be well matched with each other, Try to avoid leaving a module/unit idle or blocked. To this end, the IDU 408 controls the Wram 412 to start loading the Winograd weight i from the DRAM 304 according to the synchronization command before the transforming data i is completely generated. The time point for starting the loading of the Winograd weights i can be determined according to the input and output bandwidth of the computing device 301 . Preferably, it can be selected at the time point T ₃ , that is, starting the forward transformation and starting the loading of the Winograd weights i at the same time. Suppose that the Winograd weight i is also downloaded at time point T4 _.

Before the Winograd weight i is loaded, at the time point T ₅ , the IDU 408 controls the MAC 413 according to the synchronization command to start the alignment multiply-accumulate operation on the positive transformation data i and the Winograd weight i to generate the alignment multiplication data i. Since the time point T5, when the _Wram 412 loads the Winograd weight i, the MAC 413 also performs the multiplication-accumulation operation, and the multiplication by the data i will be completed at the time point _T6 .

Before the generation of the bit-multiplied data i is completed, at the time point T ₇ , the IDU 408 controls the ITU 415 to start reading the bit-multiplied data i from the WRram 414 according to the instruction to perform inverse transformation to generate the convolution result i. Since the time point _T7 , while the MAC 413 is performing the bitwise multiply-accumulate operation, the ITU 415 is also performing the inverse transform operation, and the convolution result i will be completed at the time point _T8 .

There can be two time points when Rram 416 starts to temporarily store the convolution result i, one is before the convolution result i is completely generated, that is, between the time points T ₇ and T ₈ , and the other is when the convolution result i is completely generated. after generation. FIG. 7 is an example of starting the temporary storage of the convolution result i after the convolution result i is completely generated. At the time point T ₈ , the IDU 408 controls the Rram 416 to start the temporary storage of the convolution result i according to the synchronization command, and at the time point T ₉ Complete staging.

After the neuron data i is output, the i+1th convolution operation can be started. The IDU 408 sends an instruction to control the Nram 409 to start loading the neuron data i+ ₁ from the DRAM 304 at the time point T2. The neuron data i+ ₁ will be completed at the time point T10, in other words, before the positive transformation data i is completely generated, the Nram 409 has started to load the neuron data i+1. Before the neuron data i+1 is loaded, at time point T ₄ , the IDU 408 controls the NTU 410 to start reading the neuron data i+1 from the Nram 409 to perform forward transformation according to the synchronization command to generate the forward transformation data i+1. Since the time point T4, while the _Nram 409 loads the neuron data i+1, the NTU 410 also reads the neuron data i+1 from the Nram409 and performs positive transformation, and the positive transformation data i+1 will be at the time point _T11 . Finish.

The IDU 408 controls the Wram 412 to start loading the Winograd weight i+1 from the DRAM 304 according to the synchronization instruction before the positive transformation data i+1 is completely generated, and before the MAC 413 completely generates the bitwise multiplication data i. The time point for starting the loading of the Wi+1 nograd weight i+1 can be determined according to the input and output bandwidth of the computing device 301. Preferably, it can be selected at the time point T ₄ , that is, starting the forward transformation and starting the loading of the Winograd weight i +1 for both being executed. Assume that the Winograd weight i+1 is also downloaded at time point _T11 .

Before the Winograd weight i+1 is loaded, at the time point T ₆ , before the ITU 415 completely generates the convolution result i, the IDU 408 controls the MAC 413 according to the synchronization command to start aligning the data i+1 and the Winograd weight i +1 Performs the bitwise multiply-accumulate operation to generate the bitwise multiplication data i+1. Since the time point _T6 , when the Wram 412 loads the Winograd weight value i+1, the MAC 413 also performs the registration multiplication and accumulation operation, and the registration multiplication data i+1 will be completed at the time point _T12 .

Before the generation of the bit-multiplied data i+1 is completed, at the time point T ₈ , the IDU 408 controls the ITU 415 to start reading the bit-multiplied data i+1 from the WRram 414 according to the instruction to perform inverse transformation to generate the convolution result i+1 . Since the time point T8, while the MAC 413 is performing the bitwise multiply-accumulate operation, the _ITU415 is also performing the inverse transform operation, and the convolution result i+1 will be completed at the time point _T13 .

Similarly, there can be two time points when Rram 416 starts to temporarily store the convolution result i+1, one is before the convolution result i+1 is completely generated, that is, between the time points T ₉ and T ₁₃ , the other is It is after the convolution result i+1 is completely generated, taking the example of starting the temporary storage of the convolution result i+1 after the convolution result i+1 is completely generated, at the time point T ₁₃ , the IDU 408 controls the Rram 416 to start up according to the synchronization command The convolution result i+1 is temporarily stored, and the temporary storage is completed at time point _T14 .

Based on the structure of the computing device 301 shown in FIG. 4 , in this embodiment, the Winograd convolution operation is performed according to the aforementioned pipeline, which can fully utilize the advantages of hardware and improve the input-output and operation efficiency.

The invention carries out hardware design based on the characteristics of the Winograd algorithm to realize the universality of acceleration, and proposes a pipeline-level operation mode to accelerate the operation speed of the Winograd convolution. In the process of hardware implementation, time division multiplexing, broadcast routing and other methods are used to make full use of the reproducible Use resources. The hardware structure proposed by the invention can match the Winograd convolution algorithm, and has the technical effects of ensuring network accuracy, performance acceleration, area reduction and power consumption reduction.

According to different application scenarios, the electronic device or device of the present invention may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC equipment, IoT terminals, mobile Terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and/or medical equipment. The vehicles include airplanes, ships and/or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lamps, gas stoves, and range hoods; the medical equipment includes nuclear magnetic resonance instruments, B-ultrasound and/or electrocardiograph. The electronic device or device of the present invention can also be applied to the Internet, Internet of Things, data center, energy, transportation, public management, manufacturing, education, power grid, telecommunications, finance, retail, construction site, medical care and other fields. Further, the electronic device or device of the present invention can also be used in application scenarios related to artificial intelligence, big data and/or cloud computing, such as cloud, edge terminal, and terminal. In one or more embodiments, the electronic device or device with high computing power according to the solution of the present invention can be applied to a cloud device (such as a cloud server), while the electronic device or device with low power consumption can be applied to a terminal device and/or Edge devices (such as smartphones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and/or the edge device are compatible with each other, so that the hardware resources of the cloud device can be retrieved from the hardware information of the terminal device and/or the edge device according to the hardware information of the terminal device and/or the edge device. Match the appropriate hardware resources to simulate the hardware resources of terminal devices and/or edge devices, so as to complete the unified management, scheduling and collaborative work of device-cloud integration or cloud-edge-device integration.

It should be noted that, for the purpose of simplicity, the present invention expresses some methods and their embodiments as a series of actions and their combinations, but those skilled in the art can understand that the solution of the present invention is not limited by the sequence of the described actions . Accordingly, based on the disclosure or teachings of the present invention, those skilled in the art will understand that some of the steps may be performed in other orders or simultaneously. Further, those skilled in the art can understand that the embodiments described in the present invention may be regarded as optional embodiments, that is, the actions or modules involved therein are not necessarily necessary for the realization of one or some solutions of the present invention. In addition, according to different solutions, the present invention also has different emphases in the description of some embodiments. In view of this, those skilled in the art can understand the parts that are not described in detail in a certain embodiment of the present invention, and can also refer to the related descriptions of other embodiments.

In terms of specific implementation, based on the disclosure and teaching of the present invention, those skilled in the art can understand that the several embodiments disclosed in the present invention can also be implemented in other ways not disclosed herein. For example, as for each unit in the foregoing electronic device or apparatus embodiment, it is divided on the basis of considering the logical function, and there may also be other division methods in actual implementation. As another example, multiple units or components may be combined or integrated into another system, or some features or functions of a unit or component may be selectively disabled. As far as the connection relationship between different units or components is concerned, the connections discussed above in conjunction with the accompanying drawings may be direct or indirect couplings between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection utilizing an interface, where the communication interface may support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

In the present invention, units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units. The aforementioned components or elements may be co-located or distributed over multiple network elements. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the solutions described in the embodiments of the present invention. In addition, in some scenarios, multiple units in this embodiment of the present invention may be integrated into one unit or each unit physically exists independently.

In other implementation scenarios, the above-mentioned integrated units may also be implemented in the form of hardware, that is, specific hardware circuits, which may include digital circuits and/or analog circuits, and the like. The physical implementation of the hardware structure of the circuit may include, but is not limited to, physical devices, and the physical devices may include, but are not limited to, devices such as transistors or memristors. In view of this, various types of devices described herein (eg, computing devices or other processing devices) may be implemented by suitable hardware processors, such as central processing units, GPUs, FPGAs, DSPs, ASICs, and the like. Further, the aforementioned storage unit or storage device can be any suitable storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), which can be, for example, a variable resistance memory (Resistive Random Access Memory, RRAM), dynamic Random Access Memory (Dynamic Random Access Memory, DRAM), Static Random Access Memory (Static Random Access Memory, SRAM), Enhanced Dynamic Random Access Memory (Enhanced Dynamic Random Access Memory, EDRAM), High Bandwidth Memory (High Bandwidth Memory, HBM), hybrid memory cube (Hybrid Memory Cube, HMC), ROM and RAM, etc.

The foregoing can be better understood in accordance with the following terms:

Clause A1. An apparatus for computing Winograd convolutions, connected to off-chip memory, the off-chip memory storing neuron data and a plurality of instructions, the apparatus comprising: a neuron cache for enabling activation from the off-chip memory Loading the neuron data into the memory; and a forward transformation unit, used to start reading the neuron data from the neuron cache to perform forward transformation before the neuron data is loaded, so as to generate forward transformation data .

Clause A2. The apparatus of Clause A1, wherein the off-chip memory further stores Winograd weights, the apparatus further comprises a weight cache for starting from the slice before the forward transformation data is completely generated Load the Winograd weights into external memory.

Clause A3. The apparatus of clause A2, wherein initiating a positive transformation is performed concurrently with initiating loading of the Winograd weights.

Clause A4. The apparatus according to Clause A2, further comprising a parametric multiply-accumulate operator for starting parametric multiplication of the positive transformed data and the Winograd weight before the Winograd weight is loaded. Accumulate operation to produce bitwise multiply data.

Clause A5. The apparatus according to Clause A4, further comprising an inverse transform unit for inversely transforming the pairwise multiplication data to generate a convolution result before the pairwise multiplication data is completely generated.

Item A6. The apparatus according to Item A5, further comprising a convolution result buffer, configured to start temporarily storing the convolution result before the convolution result is completely generated.

Clause A7. The apparatus according to Clause A5, further comprising a convolution result buffer, for starting to temporarily store the convolution result after the convolution result is completely generated.

Clause A8. The apparatus of Clause A5, wherein before the inverse transform unit fully generates the convolution result, the bitwise multiply-accumulate operator initiates an alignment of the next forward transformed data with the next Winograd weight. A bitwise multiply-accumulate operation to generate the next bitwise multiply data.

Clause A9. The apparatus of Clause A4, wherein the weight cache is enabled to load the next Winograd weights from the off-chip memory before the pairwise multiply-accumulate operator fully generates the pairwise multiply data .

Clause A10. The apparatus of Clause A1, wherein the neuron cache starts to load the next neuron data from the off-chip memory before the forward transform unit fully generates the forward transform data.

Clause A11. An integrated circuit device comprising the device of any one of clauses A1 to 10.

Clause A12. A board comprising the integrated circuit device of clause A11.

The embodiments of the present invention have been introduced in detail above, and specific examples are used to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present invention; at the same time, for Persons of ordinary skill in the art, according to the idea of the present invention, will have changes in the specific embodiments and application scope. To sum up, the contents of this specification should not be construed as limiting the present invention.

Claims (12)

1. a device for calculating Winograd convolution is connected to off-chip memory, and the off-chip memory stores neuron data and a plurality of instructions, and the device comprises: a neuron cache to enable loading of the neuron data from the off-chip memory; and The forward transformation unit is used to start reading the neuron data from the neuron buffer to perform forward transformation before the neuron data is loaded, so as to generate forward transformation data. 2 . The apparatus according to claim 1 , wherein the off-chip memory further stores Winograd weights, the apparatus further comprises a weight cache, which is used to start from the slice before the positive transformation data is completely generated. 3 . Load the Winograd weights into external memory. 3. The apparatus of claim 2, wherein initiating forward transformation is performed simultaneously with initiating loading of the Winograd weights. 4. The apparatus according to claim 2, further comprising a parametric multiply-accumulate operator, used to start the parametric multiplication of the positive transformed data and the Winograd weight before the Winograd weight is loaded. Accumulate operation to generate bitwise multiply data. 5 . The apparatus according to claim 4 , further comprising an inverse transformation unit for inversely transforming the bitwise multiplication data to generate a convolution result before the bitwise multiplication data is completely generated. 6 . 6. The apparatus according to claim 5, further comprising a convolution result buffer, for starting to temporarily store the convolution result before the convolution result is completely generated. 7. The apparatus according to claim 5, further comprising a convolution result buffer for starting to temporarily store the convolution result after the convolution result is completely generated. 8. The apparatus according to claim 5, wherein before the inverse transform unit completely generates the convolution result, the bitwise multiply-accumulate operator starts to compare the next positive transform data with the next Winograd weight value A bitwise multiply-accumulate operation to generate the next bitwise multiply data. 9. The apparatus of claim 4, wherein the weight cache is enabled to load the next Winograd weight from the off-chip memory before the para-multiply-accumulate operator fully generates the para-multiply data . 10. The apparatus of claim 1, wherein the neuron cache starts to load next neuron data from the off-chip memory before the forward transform unit fully generates the forward transform data. 11. An integrated circuit device comprising the device of any one of claims 1 to 10. 12. A board comprising the integrated circuit device of claim 11.

Images