CN106250103A - A kind of convolutional neural networks cyclic convolution calculates the system of data reusing - Google Patents

Abstract

The invention discloses a coarse-grained reconfigurable system-oriented convolution neural network loop convolution calculation data reuse system, including a main controller and a connection control module, an input data reuse module, a convolution loop operation processing array, and a data transmission Pathways in four parts. During the convolution cycle operation, the essence is to multiply multiple two-dimensional input data matrices with multiple two-dimensional weight matrices. Generally, these matrices are large in size, and the multiplication takes up most of the time of the entire convolution calculation. The present invention utilizes a coarse-grained reconfigurable array system to complete the convolution calculation process. After receiving the convolution operation request instruction, the register rotation method is used to fully explore the reusability of the input data in the convolution cycle calculation process, and the data utilization rate is improved. It also reduces the bandwidth access pressure, and the designed array unit is configurable, and can complete convolution operations with different cyclic convolution scales and step sizes.

Description

A Convolutional Neural Network Circular Convolution Computational Data Reuse System

technical field

The invention relates to the field of embedded reconfigurable design, in particular to a system for reusing convolutional neural network cyclic convolution calculation data for coarse-grained reconfigurable systems, which can be used in high-performance reconfigurable systems to realize convolutional neural networks Carry out a large number of circular convolution operations, use existing data as much as possible, reuse data, improve operation speed, and reduce data reading bandwidth pressure.

Background technique

Reconfigurable processor architecture is an ideal platform for application acceleration. Because the hardware structure can be reorganized according to the data flow graph of the program, reconfigurable arrays have been proved to have good performance improvement potential for scientific computing or multimedia applications.

Convolution operation has a wide range of uses in the field of image processing. For example, convolution operation is used in image filtering, image enhancement, image analysis and other processing. Image convolution operation is essentially a matrix operation, which is characterized by a large amount of calculation. , and the data reuse rate is high, it is difficult to use software to calculate image convolution to meet the real-time requirements.

As a feed-forward multi-layer neural network, convolutional neural network can automatically learn a large amount of labeled data and extract complex features from it. The advantage of convolutional neural network is that it only needs less preprocessing of the input image. The visual pattern is recognized in the pixel image, and it also has a good recognition effect on the recognition objects with many changes. At the same time, the recognition ability of the convolutional neural network is not easily affected by the distortion of the image or simple geometric transformation. As an important direction of multi-layer artificial neural network research, convolutional neural network has been a research hotspot for many years.

If the convolution template is placed in the upper left corner of the image lattice, the convolution template must coincide with the segmentation matrix in the upper left corner of the image lattice. Multiply their coincident items correspondingly, and then sum them all up to get the first result point. Then, move the convolution template to the right by one column to obtain the second result point. In this way, the convolution template traverses the image lattice once, and the convolution of a frame of image can be completely calculated. The data reuse rate is very high, but the traditional way of caching or directly reading from the outside is limited by the data reading bandwidth and there is no configurable array to complete the multi-layer convolution cycle operation, which is inefficient.

Contents of the invention

Purpose of the invention: Aiming at the problems and deficiencies in the prior art, the present invention provides a system for reusing convolutional neural network data for coarse-grained reconfigurable systems, which can accelerate the completion of large-scale convolution calculations. , reducing the pressure on the broadband, and the array of convolution operations is configurable. The computing performance of convolutional neural network and the occupation of hardware resources are two aspects that need to be compromised in the realization of coarse-grained reconfigurable system of convolutional neural network. The design goal of convolutional neural network based on reconfigurable processing array is Under the premise of meeting the application performance requirements, make full use of the computing resources and storage resources provided by the reconfigurable array, use the input image data reuse structure, use the high reuse rate in the circular convolution operation, and the reconfigurable coarse-grained array Configurability, in the case of limited data read bandwidth and computing resources, the convolution calculation is completed to achieve a better compromise.

Technical solution: A convolutional neural network cyclic convolution calculation data reuse system for coarse-grained reconfigurable systems, including a main controller and connection control module, input data reuse module, convolution cyclic operation processing array and data transmission path .

The main controller and the connection control module complete the reception of the external convolution operation request, load the calculation array configuration information, return the calculation result and monitor the cycle operation status, and control the data transmission between the external memory and the input data reuse module.

The input data reuse module is a data reuse module connected between the external input data memory and the circular convolution operation processing array to complete the input data reuse, wherein the upper part of the module is the image matrix width quantity FIFO, and the lower part is the image matrix Width Quantity Shift Register. FIFO continuously loads input data from the external memory, corresponding to a column of convolution calculation. When the shift register moves according to the convolution step size, FIFO replaces one of the columns for the shift register, and then completes a convolution operation to achieve the effect of data reuse. A shift register is used to provide updated neighborhood data using the upper half FIFO section. Since multiple shift registers adopt ring addressing mode, the data from FIFO will always replace the oldest data in the ring shift register, and then transfer the data to the operation array to complete the convolution operation.

The specific steps to implement this module are as follows:

The data is input S (1<=S<maximum image matrix width) pieces of 32-bit data to FIFO at one time. When the convolution operation uses the data in a register, FIFO will transfer its own data to the shift register. The shift register needs Update a column of K (1<=K<maximum image matrix width, K is the convolution kernel matrix width for this convolution calculation) 32-bit data, plus the original K-1 column data, the shift register will K*K data Transfer to the convolution calculation matrix, and then continue to move backward according to the step size. Also only need to update one column to realize the reuse of input data.

The circular convolution operation processing array obtains the required input data from the input data reuse module, completes the convolution calculation, and sends the data out after the calculation is completed.

The data transmission channel is a data transmission channel between the main controller and the interface control module, the circular convolution operation processing array, and the input data reuse module.

Further, the main controller and the connection control module include the main control and the connection controller. The connection controller has the functions of prefetch judgment and data reuse configuration control. The prefetch judgment is used to judge whether the data required for the convolution operation is ready. If the data is in place, the circular convolution operation processing array performs the convolution cycle calculation, if not, it waits for the data to be in place. The data in the cache is read from the external memory, and the present invention adopts direct memory access mode to read. When external data input is required, the main controller sends a command to read data from the external memory, and then the main controller does not store data. Read control, the connection controller will send a stop signal to the main controller, the main controller will give up the right to use the address bus, data bus and related control bus, and when the data of the input data reuse module needs to be updated, it will pass the connection control device, directly read the data in the external memory.

The circular convolution operation processing array includes an array configuration module, including an array configuration module, a storage processing unit, and a calculation processing unit. Configuration, using the available computing resources of the array, reconfigure the array after each calculation is completed, and the calculation processing unit is adjusted according to the calculation scale to perform the next convolution operation.

The convolution operation processing array configuration controller, after the interface control module loads the configuration information, the operation array can make the convolution image matrix scale variable from 1 to the maximum image matrix according to the size of the circular convolution cycle scale and the step size information For the value calculation between widths, each convolution operation can reconfigure the operation array. When the convolution kernel is small, the convolution array can still use the entire convolution calculation matrix to shorten the total time of convolution calculation.

Storage and calculation unit structure The storage instruction is closely related to the data reuse module. Driven by the loop control unit, it takes the address from the address queue or directly calculates the address through the address generation unit, sends a read data request to the data reuse module, and returns the data to write Into the data queue, under the control of the loop end component, read the data in the shift register.

The calculation and processing unit realizes the calculation and selection functions in the process of data flow. The cyclic subscript continuously obtains data from the register group and transfers the data to the calculation and processing unit array. The calculation and processing unit array performs operations according to the fixed connection relationship. The result is stored to the specified location.

The cyclic convolution operation processes the continuous pipeline operation of the array application. This operation is cyclically mapped to the array configuration module. The array configuration module configures the initial value, final value and step value of the loop control variable. The execution of the loop program does not require external control. Each calculation The pipeline link is formed between the array units, and the scheduling of the circular convolution on the pipeline is completed.

Description of drawings

FIG. 1 is a schematic diagram of a coarse-grained reconfigurable array architecture for convolution calculation in an embodiment of the present invention;

FIG. 2 is a hardware structural diagram of the data round-robin scheduling of the input data reuse module in an embodiment of the present invention;

3 is a structural block diagram of a storage processing unit in a coarse-grained reconfigurable convolution computing array in an embodiment of the present invention;

4 is a structural block diagram of a coarse-grained reconfigurable convolution calculation array calculation processing unit in an embodiment of the present invention;

Fig. 5 is a flow chart of implementing circular convolution in a reconfigurable array in an embodiment of the present invention.

detailed description

Below in conjunction with specific embodiment, further illustrate the present invention, should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention, after having read the present invention, those skilled in the art will understand various equivalent forms of the present invention All modifications fall within the scope defined by the appended claims of the present application.

Coarse-grained reconfigurable system-oriented convolution neural network loop convolution calculation data reuse system, including the main controller and connection control module, input data reuse module, convolution loop operation processing array and data transmission path.

The main controller and the connection control module complete the reception of external convolution operation requests, load the calculation array configuration information, return the calculation results and monitor the cycle operation status, and control the data transmission between the external memory and the input data reuse module.

The input data reuse module is a data reuse module connected between the external input data memory and the circular convolution operation processing array, wherein the upper part of the module is the image matrix width quantity FIFO, and the lower part is the image matrix width quantity shift register.

The circular convolution operation processes the array, obtains the required input data from the input data reuse module, completes the convolution calculation, and sends the data out after the calculation is completed.

The data transmission channel is the data transmission channel between the main controller and the interface control module, the circular convolution operation processing array, and the input data reuse module.

The main controller and the connection control module include the main control and the connection controller. The connection controller has the functions of prefetch judgment and data reuse configuration control. The prefetch judgment is used to judge whether the data required for convolution operation is ready. If the data is in place, the circular convolution operation processing array performs the convolution cycle calculation, if not, it waits for the data to be in place. The data in the cache is read from the external memory, and the present invention adopts direct memory access mode to read. When external data input is required, the main controller sends a command to read data from the external memory, and then the main controller does not store data. Read control, the connection controller will send a stop signal to the main controller, the main controller will give up the right to use the address bus, data bus and related control bus, and when the data of the input data reuse module needs to be updated, it will pass the connection control device, directly read the data in the external memory.

As shown in Figure 1, the coarse-grained reconfigurable array graph of the specific computing array graph and data flow. The configurable PE unit occupies the most important part, also because the reconfigurable array is the specific part that completes the convolution calculation, and the rest is mainly to transmit the start and end instructions. It can be seen from Figure 1 that the storage processing unit in the configurable array is directly connected to the input data reuse module (as shown in Figure 2). According to the step size and convolution kernel scale information, the input data reuse module transmits the data stream required for the convolution operation to The computing processing unit and the router configure the data stream to reach each computing processing unit through the Internet route, and at the same time, the connection controller is responsible for completing a convolution calculation, transmitting the data information, and reconfiguring the computing processing unit to start the next new operation.

The data round-robin scheduling hardware diagram of the input data reuse module is shown in Figure 2. Taking the convolution kernel size as K*K (K is the convolution kernel width) as an example, a FIFO is added between the external memory and the shift register. The data is input S pieces of 32-bit data to the FIFO at a time. When the convolution operation uses the data in a register, the FIFO will transfer its own data to the shift register. The shift register needs to update a column of K 32-bit data, plus the original K-1 columns of data, the shift register transmits K*K data to the convolution calculation matrix. This kind of input image data reuse structure provides support for high-efficiency convolution operations.

As shown in Figure 3, it corresponds to the structural block diagram of the storage processing unit. When the input channel receives the address signal, it corresponds to the position of the storage processing unit in the array. These storage processing units complete the generation of the address of the corresponding data. When the address is generated, it will correspond to the data in the input image data reuse module, and then output the data to the computing processing unit. The loop controls the generation of the address corresponding to the operation data, and the end of the convolution operation, and synchronously transmits the calculated data to the external memory. Moreover, when the data in the loop judging structure is wrong or insufficient, the current operation is ended, and the information is transmitted to the external memory for data update.

As shown in Figure 4, it corresponds to the structure diagram of the calculation processing unit. When the calculation processing unit receives the input data, it uses the internal multiplier and adder to complete the convolution operation, completes an operation, and reconfigures the operation according to the configuration controller. The required calculation and processing unit can complete the configurable control, and when the size and step size of the outer loop are changed, the operation can still be completed well.

Combined with Figure 1 and Figure 2, the specific steps of convolution cycle calculation are shown in Figure 5, including the following steps:

1) If the coarse-grained reconfigurable array system is required to complete a large number of convolution operations, it must first send a request to the convolution control system. When the main processor receives the request, it will send instructions to the connection processing unit;

2) The connection processing unit first judges whether the required data in the input data reuse module is in place, if not, it will send a waiting signal, and at the same time use direct storage access to transfer data to the buffer;

3) After the data is completed, the waiting operation instruction is notified, the control cycle starts, the configuration control unit in the convolution cycle operation processing array will configure the array, and the memory access configuration module in the calculation array will calculate the location of the data. position, and then the calculation array performs convolution calculation on the data at this position, which is performed sequentially.

4) Y (maximum image matrix width) FIFO buffers continuously update the used data in the register through direct storage and reading. When entering this position again, the data has been updated, and the operation is performed without interruption, and there is no need for each convolution Operation to external storage to access data.

5) The control cycle of the connection controller is completed. When the calculation is completed, the final data is output to the external memory, and the convolution operation array is completed this time.

When performing a large number of circular convolution operations, when computing resources are limited, the method of data reuse is applied, coupled with configurable and reconfigurable arrays, and the pipeline completes convolution operations, which improves the efficiency and speed of operations. A comparative test is set up, which are comparative verification system A and comparative verification system B respectively. Among them, system A is compared and verified, that is, a traditional reconfigurable system that does not support array configuration and reuse. Compare and verify system B, which is the reconfigurable system that supports data prefetching and reuse proposed by the present invention. Select a 16x16 input data matrix, a 3x3 convolution matrix, and a step size of 1, set 10 input data, 10 convolution weight matrices, and perform convolution operations at the same time. The experimental results show that the comparison verification system B can obtain an average performance improvement of 1.76 times that of the comparison verification system A.

Claims (5)

1. A system for reusing convolutional neural network cyclic convolution computing data for coarse-grained reconfigurable systems, characterized in that it includes a main controller and a connection control module, an input data reuse module, a convolutional cyclic operation processing array and data transmission path; The main controller and the connection control module complete the reception of the external convolution operation request, load the calculation array configuration information, return the calculation result and monitor the cycle operation state, and control the data transmission between the external memory and the input data reuse module; The input data reuse module is a data reuse module connected between the external input data memory and the circular convolution operation processing array, wherein the upper part of the module is an image matrix width quantity FIFO, and the lower part is an image matrix width quantity shift register ; The circular convolution operation processing array obtains the required input data from the input data reuse module, completes the convolution calculation, and sends the data out after the calculation is completed. 2. The data transmission channel is a data transmission channel between the main controller and the interface control module, the circular convolution operation processing array, and the input data reuse module. 3. The system for reusing convolutional neural network cyclic convolution calculation data for coarse-grained reconfigurable systems as claimed in claim 1, wherein: the main controller and the connection control module include a main control and a connection controller, and the connection The controller has the functions of prefetch judgment and data reuse configuration control. The prefetch judgment is used to judge whether the data required for convolution operation is ready. If the data is in place, the circular convolution operation processing array performs convolution cycle calculation. , if not, then wait for the data to be in place; the data in the cache is read from the external memory, and the direct memory access method is used to read. When external data input is required, the main controller sends a read data command, after which the main controller will not control the memory reading, the connection controller will send a stop signal to the main controller, the main controller will give up the right to use the address bus, data bus and related control bus, and input data reuse module When the data needs to be updated, it directly reads the data in the external storage by connecting to the controller. 4. The system for reusing convolutional neural network cyclic convolution calculation data for coarse-grained reconfigurable systems as claimed in claim 1, wherein the cyclic convolution operation processing array includes an array configuration module, including an array configuration module, Storage processing unit and calculation processing unit. This module is used to match the input data reuse module. According to the convolution calculation scale and step size, the array configuration module configures the calculation array and uses the available computing resources of the array. After each calculation is completed once The array is reconfigured, and the calculation processing unit is adjusted according to the calculation scale, and the next convolution operation is performed; the circular convolution operation processes the continuous pipeline operation of the array application, and this operation is cyclically mapped to the array configuration module, and the array configuration module configures the initial value of the loop control variable. Value, final value, and step value, the execution of the cyclic program does not require external control, and a pipeline link is formed between each calculation array unit to complete the scheduling of the cyclic convolution on the pipeline. 5. The system for reusing convolutional neural network cyclic convolution calculation data for coarse-grained reconfigurable systems as claimed in claim 1, wherein: the input data reuse module realizes specific steps as follows: The data is input S pieces of 32-bit data to the FIFO at a time. When the convolution operation uses the data in a register, the FIFO will transfer its own data to the shift register. The shift register needs to update a column of K 32-bit data, plus the original K-1 columns of data, the shift register transmits K*K data to the convolution calculation matrix, and then continues to move backward according to the step size, and only needs to update one column to realize the reuse of input data.