WO2021243839A1 - Composite-granularity, near-storage and approximation-based acceleration structure and method for long short-term memory network - Google Patents

Abstract

The present invention relates to the field of acceleration techniques for long short-term memory networks. Provided are a composite-granularity, near-storage and approximation-based acceleration structure and method for a long short-term memory network. The invention mainly employs a composite-granularity-based division policy to perform parallel division with respect to computation tasks. The acceleration structure comprises a matrix vector operation module, a near-storage and approximation-based acceleration storage module, a near-storage and approximation-based acceleration operation module, and a function configuration module for near-storage and approximation-based acceleration operations. The composite-granularity, near-storage and approximation-based acceleration structure and method for a long short-term memory network closely couple a storage structure with an approximation-based computation unit structure, and employ a composite-granularity-based task division and parallel computation policy to provide a highly efficient and flexible acceleration structure for a long short-term memory neural network.

Description

Approximate acceleration structure and method for compound granularity near storage of long and short-term memory network Technical field

The invention belongs to the technical field of long- and short-term memory network acceleration, and specifically relates to a compound-granularity near-storage approximate acceleration structure and method of a long- and short-term memory network.

Background technique

In recent years, with the continuous development of deep learning, long short-term memory network (LSTM), as a special recurrent neural network, has been widely used in many fields because of its very good performance on long sequences, such as sound, Video etc. However, with the rapid increase in neural network applications, the continuous expansion of network scale, the sharp increase in the data flow that needs to be processed, and the further increase in processing delay and power consumption requirements, will bring great challenges to memory and bandwidth, and at the same time The dependence between data and centralized computing requirements greatly limit the performance of network accelerators. This makes it difficult for the long and short-term memory network of the traditional structure to meet the design requirements.

Therefore, in view of the problems of high memory bandwidth requirements and high computational power consumption in the long- and short-term memory network, it is necessary to improve the existing technology to improve the processing parallelism and operation speed of the long- and short-term memory neural network.

Summary of the invention

The purpose of the present invention is to overcome the deficiencies in the prior art and propose a compound granular near storage approximate acceleration structure and method for long and short-term memory networks, based on the compound granular network model to achieve the division of long and short-term memory network scheduling strategies, and use near Storing approximate acceleration calculation modules to perform calculations can better improve the parallelism and speed of calculations.

In order to solve the above technical problems, the present invention proposes the following technical solutions:

The present invention proposes a compound granularity near-storage approximate acceleration structure of a long and short-term memory network based on a compound granularity task division strategy. The composite granularity is composed of coarse granularity and fine granularity. Coarse granularity is the parallel acceleration at the cell level, and fine granularity is the acceleration of the matrix inside the cell.

The compound granularity near-storage approximate acceleration structure of the long-short-term memory network proposed by the present invention includes: a near-storage approximate acceleration storage module, a matrix vector operation module, a near-storage approximate acceleration operation module, and a functional configuration module for near-storage approximate acceleration operation.

First, the matrix-vector operation module is used to calculate between the matrix and the vector. The calculated intermediate vector data is stored in the near-storage approximate acceleration storage module, and the near-storage approximate acceleration storage module provides various calculations to the near-storage approximate acceleration calculation module. For vector data, the near-storage approximate acceleration calculation module is used to perform vector-to-vector calculations, and the near-storage approximate acceleration calculation function configuration module is used to configure the near-storage approximate acceleration calculation module.

Further, the vector-to-vector calculation tasks performed by the near-storage approximate acceleration calculation module include several different vector calculation types, and the near-storage approximate acceleration calculation function configuration module supports different vector calculation types.

Furthermore, the matrix vector operation module is mainly used to calculate multiplication and addition operations, and the near storage approximate acceleration operation module is used to calculate activation functions or addition operations.

For the long and short-term memory network, the composite granular task division strategy is used to divide the calculation tasks, so that the calculation tasks between the matrix and the vector are sent to the matrix vector operation module, and the calculation tasks between the vector and the vector are sent to the near storage approximate acceleration operation Modules, and two computing modules parallel computing tasks at the same time, so as to achieve computing acceleration, making the execution efficiency higher and the power consumption lower.

In the composite granularity near-storage approximate acceleration structure of the long-short-term memory network proposed by the present invention, the near-storage approximate acceleration calculation module includes: a first data storage module, a second data storage module, and a data processing unit. The data that needs to be calculated are respectively input into the first data storage module and the second data storage module, and the data obtained by the calculation is output by the first data storage module.

Both the first data storage module and the second data storage unit module are address storage areas with a size of 1KB, and both have a bit width of 16×16 bits and a depth of 32. The first part and S _0i (i=1, 2, ... 9, A, B), the second part and S _1i (i=1, 2, ... 9, A, B) are respectively stored in the first data storage module And the second data storage module.

Further, based on the compound granularity task division strategy, the _{calculation steps of the first part and S 0i} , the second part and S _{1i of} LSTM are as follows:

Step A01, suppose that at the t-th time, the network reads the t-th input x _t , and configures the input gate i, the forget gate f, the memory unit c, and the output gate o response weight b and offset value W. The first part and , The second part and meet the following announcements:

Step A02: Calculate the response values of input gate i, forget gate f, memory unit c, and output gate o. The network reads the state value h _{t-1 of the} hidden layer at time t-1, and the first part and the second part are satisfied The following announcement:

Step A03: Calculate the response values of input gate i, forget gate f, memory cell c and output gate o, the network reads the memory cell vector value c _t-1 at time t-1, the first part and the second part satisfy the following Publicity:

Step A04: Calculate the response values of the input gate i, the forget gate f, the memory unit c, and the output gate o. The sum of the first part and the second part meet the following announcements:

Step A05: Calculate the vector values i _t and f _{t of the} input gate i and the forget gate f, and calculate the response values of the memory unit c and the output gate o. The sum of the first part and the second part meet the following announcements:

In the above _{formula, i t = σ (W ix} x t + W ih h t-1 + W ic c t-1 + b i), f t = σ (W fx x t + W fh h t-1 + W _fc c _t-1 +b _f ), where σ is the sigmoid function.

Step A06: Calculate the response values of the memory unit c and the output gate o, and the first part sum and the second part sum meet the following announcements:

In step A07, the vector value c _{t of the} memory unit c is calculated, and the response value of the output gate o is calculated. The sum of the first part and the second part satisfy the following announcements:

In the above formula, c _t =f _t ⊙c _t-1 +i _t ⊙φ(W _cx x _t +W _ch h _t-1 +b _c ), where ⊙ represents the element-wise multiplication operation, and φ is the hyperbolic tangent function .

Step A08, calculate the response value of the output gate o, the first part and the second part meet the following announcements:

Step A09, calculate the response value of the output gate o, the first part and the second part meet the following announcements:

Step A10, calculate the vector value of the output gate o, the first part sum and the second part sum satisfy the following publicity:

In the above formula, o _t =σ(W _ox x _t +W _oh h _t-1 +W _oc c _t-1 +b _o ).

Step A11, the state value h _{t of the} hidden layer at time t is calculated, and the first part sum and the second part sum satisfy the following publicity:

In the above formula, h _t =o _t ⊙φ(c _t ).

The data processing unit in the near storage approximate acceleration calculation module includes: a configuration file cache, a configuration file parser, a first address generator, a second address generator, a multiplexer, and a multifunctional array processor.

The configuration environment file is loaded into the configuration file cache, and then the configuration file parser performs the address configuration analysis operation, and the obtained address configuration files are respectively loaded into the first address transmitter and the second address generator, and the first address generator is based on the address configuration file Decide whether to select the corresponding first address from the first data storage module, and the second address generator determines whether to select the corresponding second address from the second data storage module according to the address configuration file, which is poured into the first address generator The first address and the second address poured into the second address generator are both input into the multiplexer; the configuration file parser also configures the multiplexer, and the multiplexer finally starts from the first place and the second address. The data corresponding to one of the two addresses is selected as the output data of the multiplexer, and is input to the multi-function array processor together with the first address; at the same time, the configuration file parser also performs the processing on the multi-function array processor. Calculate the configuration, calculate the output data of the multiplexer, and then store the calculation result in the first address.

Further, the configuration file cache is a cache array dedicated to configuration environment files; the multi-function array processor is a reconfigurable multi-function array processor, whose input is a 16-bit fixed-point number, which can perform addition, multiplication, and sigmoid operations.

In the compound-granularity near-storage approximate acceleration structure of the long-short-term memory network proposed by the present invention, the function configuration is realized by the function configuration module of the near-storage approximate acceleration operation. The function configuration module of the near-storage approximate acceleration operation has a bit width of 16, and the function configuration modules include : Address configuration unit, multiplexer configuration unit, calculation configuration unit.

Further, bits 0 to 7 of the function configuration module are the address configuration unit; among them, bits 0 to 2 of the function configuration module are the address generator selection unit, which is used to select the address generator; Bits 3 to 7 are the address selection unit, which is used to select the address in the address generator.

Further, the 8th to 11th bits of the function configuration module are the multiplexer configuration unit, which is used to select the data for the multiplier operation.

Further, the 12th to 15th bits of the functional configuration module are the calculation configuration unit, which is used to indicate the type of operation. The calculation configuration unit can indicate addition, multiplication, logical operation, sigmoid operation, and approximate multiplication. The last calculation configuration unit Two bits are used to configure the number of iterations for approximate multiplication. The more iterations, the more accurate the calculation result will be.

The present invention also proposes a compound-granularity near-storage approximate acceleration method for a long and short-term memory network. The steps of the acceleration method are as follows:

Step S1, load the configuration file: load the configuration environment file into the configuration configuration file cache;

Step S2, parsing the loaded configuration file, specifically includes the following three parallel steps:

Step S2-1, resolve the address configuration:

The configuration file parser performs the address configuration analysis operation, and the obtained address configuration files are loaded into the first address transmitter and the second address generator respectively, and used to select whether to take the address in the first address generator or select the second address to occur Address in the device;

Further, the first address generator decides whether to select the corresponding first address from the first data storage module according to the address configuration file and fill it into the first address generator, and the second address generator decides whether to select from the second address according to the address configuration file. Select the corresponding second address in the data storage module and fill it into the second address generator;

Step S2-2, parse the multiplexer configuration:

The configuration file parser performs the multiplexer configuration analysis operation, obtains the multiplexer configuration file, and the multiplexer selects the data source;

Further, the first address that is poured into the first address generator and the second address that is poured into the second address generator are both input into the multiplexer;

After being configured, the multiplexer selects data corresponding to an address from the first address and the second address as the output data of the multiplexer, and is input into the multifunctional array processor together with the first address;

Step S2-3, analyze and calculate the configuration:

The configuration file parser performs the calculation configuration of the multi-function array processor. The reconfigurable multi-function array processor (RMPA) is calculated and configured to perform RMPA calculation on the output data of the multiplexer, and then the calculation result is stored in the first Address;

In step S3, it is judged whether there are any more configuration files, if there are any more configuration files, return to step S1, and if there are no more configuration files, then it ends.

Compared with the prior art, the near-storage approximate acceleration structure oriented to the long-short-term memory neural network proposed by the present invention has the following benefits:

The storage structure and the approximate computing unit structure are tightly coupled. Through the task division of compound granularity and the parallel computing strategy, a more efficient and flexible acceleration structure is designed for the long- and short-term memory neural network. In specific calculations, through the task division of compound granularity and parallel computing strategies, the parallelism of tasks is increased by more than 30%, and the near-memory approximate acceleration structure and method can further reduce the performance of power consumption, which can be increased by more than 20% Energy efficiency of the system.

Description of the drawings

FIG. 1 is a structural frame diagram of the near-storage approximate acceleration computing module in the compound-granularity near-storage approximate acceleration structure of the long-short-term memory network proposed by the present invention;

2 is a working flow chart of the near-storage approximate acceleration calculation module circuit in the compound-granularity near-storage approximate acceleration structure of the long and short-term memory network proposed by the present invention;

FIG. 3 is a schematic diagram of the functional configuration module structure of the near-memory approximate acceleration operation in the compound-granularity near-memory approximate acceleration structure of the long and short-term memory network proposed by the present invention.

detailed description

The present invention will be further described in detail below in conjunction with examples.

Example 1. The acceleration structure of the traditional long- and short-term memory network is designed based on a single-granularity task division strategy, while the present invention proposes a compound-granularity near-storage approximate acceleration structure of the long- and short-term memory network, which is designed based on a compound-granularity task division strategy.

Furthermore, the composite granularity is composed of coarse granularity and fine granularity. Coarse granularity is parallel acceleration at the cell level, and fine granularity is matrix acceleration inside the cell. The task division strategy based on compound granularity can break the partition relationship between the cell level and the gate level.

Based on the compound-granularity task division strategy, the specific steps of LSTM calculation are as follows:

Step A01, suppose that at the t-th time, the network reads the t-th input x _t , and configures the input gate i, the forget gate f, the memory unit c, and the output gate o response weight b and offset value W. The first part and , The second part and meet the following announcements:

Step A02: Calculate the response values of input gate i, forget gate f, memory unit c, and output gate o. The network reads the state value h _{t-1 of the} hidden layer at time t-1, and the first part and the second part are satisfied The following announcement:

Step A03: Calculate the response values of input gate i, forget gate f, memory cell c and output gate o, the network reads the memory cell vector value c _t-1 at time t-1, the first part and the second part satisfy the following Publicity:

Step A04: Calculate the response values of the input gate i, the forget gate f, the memory unit c, and the output gate o. The sum of the first part and the second part meet the following announcements:

Step A05: Calculate the vector values i _t and f _{t of the} input gate i and the forget gate f, and calculate the response values of the memory unit c and the output gate o. The sum of the first part and the second part meet the following announcements:

In the above _{formula, i t = σ (W ix} x i + W ih h t-1 + W ic c t-1 + b i), f t = σ (W fx x t + W fh h t-1 + W _fc c _t-1 +b _f ), where σ is the sigmoid function.

Step A06: Calculate the response values of the memory unit c and the output gate o, and the first part sum and the second part sum meet the following announcements:

In step A07, the vector value c _{t of the} memory unit c is calculated, and the response value of the output gate o is calculated. The sum of the first part and the second part satisfy the following announcements:

In the above formula, c _t =f _t ⊙c _t-1 +i _t ⊙φ(W _cx x _t +W _ch h _t-1 +b _c ), where ⊙ represents the element-wise multiplication operation, and φ is the hyperbolic tangent function .

Step A08, calculate the response value of the output gate o, the first part and the second part meet the following announcements:

Step A09, calculate the response value of the output gate o, the first part and the second part meet the following announcements:

Step A10, calculate the vector value of the output gate o, the first part sum and the second part sum satisfy the following publicity:

In the above formula, o _t =σ(W _ox x _t +W _oh h _t-1 +W _oc c _t-1 +b _o ).

Step A11, the state value h _{t of the} hidden layer at time t is calculated, and the first part sum and the second part sum satisfy the following publicity:

In the above formula, h _t =o _t ⊙φ(c _t ).

The parallel division strategy of computing tasks based on compound-granularity LSTM can better realize the parallelism of data-level operations under fine-grainedness and the parallelism of cell-level operations under coarse-grainedness. By reorganizing and redistributing tasks in LSTM, Improve parallelism. In the composite granular network allocation model, the same operation in different gates and cells will be executed only once. It converts dependent operations between gates and cells in the fine-grained model and between stages in the coarse-grained model into independent operations.

Example 2. The compound granularity near-storage approximate acceleration structure of the long-short-term memory network proposed by the present invention includes: a near-storage approximate acceleration storage module, a matrix vector operation module, a near-storage approximate acceleration operation module, and a functional configuration module for near-storage approximate acceleration operation.

Among them, first, the matrix vector operation module is used to calculate between the matrix and the vector, and the calculated intermediate vector data obtained is stored in the near storage approximate acceleration storage module, and the near storage approximate acceleration storage module moves to the near storage approximate acceleration calculation module. Provides various vector data. The near-storage approximate acceleration calculation module is used to perform calculations between vectors, and the near-storage approximate acceleration calculation function configuration module is used to configure the near-storage approximate acceleration calculation module.

Further, the vector-to-vector calculation tasks performed by the near-storage approximate acceleration calculation module include several different vector calculation types, and the near-storage approximate acceleration calculation function configuration module supports different vector calculation types.

Furthermore, the matrix vector operation module is mainly used to calculate multiplication and addition operations, and the near storage approximate acceleration operation module is used to calculate activation functions or addition operations.

For the long and short-term memory network, the composite granular task division strategy is used to divide the calculation tasks, so that the calculation tasks between the matrix and the vector are sent to the matrix vector operation module, and the calculation tasks between the vector and the vector are sent to the near storage approximate acceleration operation Modules, and two computing modules parallel computing tasks at the same time, so as to achieve computing acceleration, making the execution efficiency higher and the power consumption lower.

Example 3. In the compound-granularity near-storage approximate acceleration structure of the long-short-term memory network proposed in the present invention, the structure of the near-storage approximate acceleration calculation module is shown in FIG. 1. The calculation module includes: a first data storage module, a second data storage module, and data Processing unit. The data that needs to be calculated are respectively input into the first data storage module and the second data storage module, and the data obtained by the calculation is output by the first data storage module. In Figure 1, the solid line represents the data flow, and the dashed line represents the configuration flow.

The first data storage module and the second data storage unit module are both address storage areas with a size of 1KB, with a bit width of 16×16 bits and a depth of 32, corresponding to the compound granularity model. The first part and S _0i (i=1, 2, ... 9, A, B), the second part and S _1i (i=1, 2, ... 9, A, B) are respectively stored in the first data storage module And the second data storage module.

The data processing unit includes: a configuration file cache, a configuration file parser, a first address generator, a second address generator, a multiplexer, and a multifunctional array processor.

The configuration environment file is loaded into the configuration file cache, and then the configuration file parser performs the address configuration analysis operation, and the obtained address configuration files are respectively loaded into the first address transmitter and the second address generator, and the first address generator is based on the address configuration file Decide whether to select the corresponding first address Add_0_x (x=0,1,2,......,30,31) from the first data storage module, and the second address generator decides whether to select from the second data storage module according to the address configuration file Select the corresponding second address Add_1_x (x=0, 1, 2, ..., 30, 31) in the first address generator, and the first address Add_0_x of the first address generator, and the second address of the second address generator. The address Add_1_x is input to the multiplexer; the configuration file parser also configures the multiplexer, and finally the multiplexer selects the data corresponding to an address from the first address Add_0_x and the second address Add_1_x as the multiplexer. The output data of the channel selector is input to the multi-function array processor together with the first address Add_0_x; at the same time, the configuration file parser also calculates and configures the multi-function array processor, and the output data of the multiplexer Perform calculation, and then store the calculation result in the first address Add_0_x.

Further, the configuration file cache is a cache array dedicated to configuration environment files; the multi-function array processor is a reconfigurable multi-function array processor, whose input is a 16-bit fixed-point number, which can perform addition, multiplication, and sigmoid operations.

Based on the network model of compound granularity, by dividing and reorganizing the tasks in the long- and short-term memory network, the fine-grained data-level parallelism and the coarse-grained unit-level parallelism are further utilized. In the compound-granularity network partition model, the same or similar types of operations in different gates and units are processed as a task, so that the dependent operations between the gates and units in the fine-grained model and the dependencies between the stages in the coarse-grained model are treated as one task. The operation is converted to independent operation.

Example 4. The present invention also proposes a compound-granularity near-storage approximate acceleration method for a long and short-term memory network. The steps of the acceleration method are shown in Figure 2, and the details are as follows:

Step S1, load the configuration file: load the configuration environment file into the configuration configuration file cache;

Step S2, parsing the loaded configuration file, specifically includes the following three parallel steps:

Step S2-1, resolve the address configuration:

The configuration file parser performs the address configuration analysis operation, and the obtained address configuration files are loaded into the first address transmitter and the second address generator respectively, and used to select whether to take the address in the first address generator or select the second address to occur Address in the device;

Further, the first address generator decides whether to select the corresponding first address Add_0_x (x=0,1,2,...,30,31) from the first data storage module according to the address configuration file and fill it into the first address Generator, the second address generator decides whether to select the corresponding second address Add_1_x (x=0,1,2,......,30,31) from the second data storage module according to the address configuration file and fill it into the second Address generator

Step S2-2, parse the multiplexer configuration:

The configuration file parser performs the multiplexer configuration analysis operation, obtains the multiplexer configuration file, and the multiplexer selects the data source;

Further, the first address Add_0_x that is injected into the first address generator and the second address Add_1_x that is injected into the second address generator are both input into the multiplexer;

After configuration, the multiplexer selects the data corresponding to an address from the first address Add_0_x and the second address Add_1_x as the output data of the multiplexer, and is input to the multifunctional array processor together with the first address Add_0_x middle;

Step S2-3, analyze and calculate the configuration:

The configuration file parser performs the calculation configuration of the multi-function array processor. The reconfigurable multi-function array processor (RMPA) is calculated and configured to perform RMPA calculation on the output data of the multiplexer, and then the calculation result is stored in the first Address Add_0_x;

In step S3, it is judged whether there are more configuration files, if there are any more configuration files, return to step S1, and if there are none, then end.

Example 5. In the composite granularity near-storage approximate acceleration structure of the long-short-term memory network proposed by the present invention, the function configuration is realized by the function configuration module of the near-storage approximate acceleration operation, and the bit width of the function configuration module of the near-storage approximate acceleration operation is 16, as shown in Figure 3. As shown, the functional configuration module includes: address configuration unit, multiplexer configuration unit, and calculation configuration unit.

Further, bits 0 to 7 of the function configuration module are address configuration units; among them, bits 0 to 2 of the function configuration module are the address generator selection unit Bank, which is used to select the address generator; Bits 3 to 7 are the address selection unit Address, which is used to select the address in the address generator.

In this preferred embodiment, when the address generator selection unit Bank is 000, it represents the selection of the first address generator, when the address selection unit Address is 00000, it represents the selection of the first address Add_0_0, and when the address selection unit Address is 00001, it represents the selection of the first address. Add_0_1, and so on, when the address selection unit Address is 11111, it means that the first address Add_0_31 is selected.

In this preferred embodiment, when the address generator selection unit Bank is 001, it represents the selection of the second address generator, when the address selection unit Address is 00000, it represents the selection of the second address Add_1_0, and when the address selection unit Address is 00001, it represents the selection of the second address. Add_1_1, and so on, when the address selection unit Address is 11111, it means that the second address Add_1_31 is selected.

Further, the 8th to 11th bits of the function configuration module are the multiplexer configuration unit MUX, which is used to select the data of the multiplier operation.

Further, the 12th to 15th bits of the functional configuration module are the calculation configuration unit OpCode, which is used to indicate the type of operation to be performed. In this preferred embodiment, the calculation configuration unit OpCode is 0000 for addition, and the calculation configuration unit OpCode is 0100 for multiplication. The calculation configuration unit OpCode is 1000 for logical operation, and the calculation configuration unit OpCode is 1100 for sigmoid operation; when it represents approximate multiplication, the last two digits of the configuration unit OpCode are used to configure the number of iterations of the approximate multiplication, and the calculation configuration unit OpCode is 0100 , 0101, 0110 indicate the number of iterations are 0, 1, and 2, respectively. The more iterations, the more accurate the calculation result.

The above specific implementations and examples are specific support for the technical idea of the composite granularity near storage approximate acceleration structure and method of the long-short-term memory network proposed by the present invention, and cannot be used to limit the protection scope of the present invention. Any technology proposed in accordance with the present invention Thinking, any equivalent changes or equivalent modifications made on the basis of this technical solution still belong to the protection scope of the technical solution of the present invention.

Claims (6)

The compound granularity near storage approximate acceleration structure of the long and short-term memory network includes: near storage approximate acceleration storage module, matrix vector operation module, near storage approximate acceleration operation module, and near storage approximate acceleration operation functional configuration module, which is characterized by: The compound granularity near storage approximate acceleration structure is based on the task division strategy of compound granularity to carry out calculation tasks in parallel, so that the calculation tasks between the matrix and the vector are sent to the matrix vector operation module, and the calculation tasks between the vector and the vector are sent to the near The storage approximate acceleration calculation module, the matrix vector calculation module, and the near storage approximate acceleration calculation module simultaneously carry out calculation tasks in parallel; the matrix vector calculation module is mainly used to calculate multiplication and addition operations, and the near storage approximate acceleration calculation module is used to calculate the activation function Or addition operation; The compound granularity is composed of coarse granularity and fine granularity, coarse granularity is cell-level parallel acceleration, and fine granularity is matrix acceleration inside the cell; The near storage approximate accelerated calculation module includes: a first data storage module, a second data storage module, and a data processing unit; The data storage module stores the first part sum, the second data storage module stores the second part sum, and the data obtained by the operation is output by the first data storage module; The functional configuration module for near-storage approximate accelerated operation includes: an address configuration unit, a multiplexer configuration unit, and a calculation configuration unit; First, the matrix-vector operation module is used to calculate between the matrix and the vector. The calculated intermediate vector data is stored in the near-storage approximate acceleration storage module, and the near-storage approximate acceleration storage module provides various calculations to the near-storage approximate acceleration calculation module. This kind of vector data, the function configuration module of the near storage approximate acceleration operation is used to configure the function of the near storage approximate acceleration operation module. The compound-granularity near-storage approximate acceleration structure of the long-short-term memory network according to claim 1, wherein the first data storage module and the second data storage unit module are both address storage areas with a size of 1KB. The width is 16×16bit, and the depth is 32. The compound-granularity near-storage approximate acceleration structure of the long-short-term memory network according to claim 1, wherein the data processing unit includes: a configuration file cache, a configuration file parser, a first address generator, and a second address generator. Processor, multiplexer, multi-function array processor; The configuration environment file is loaded into the configuration file cache, and then the configuration file parser performs the address configuration analysis operation, and the obtained address configuration files are respectively loaded into the first address transmitter and the second address generator, and the first address generator is based on the address configuration file Decide whether to select the corresponding first address from the first data storage module, and the second address generator determines whether to select the corresponding second address from the second data storage module according to the address configuration file, which is poured into the first address generator The first address and the second address poured into the second address generator are both input into the multiplexer; the configuration file parser also configures the multiplexer, and the multiplexer finally starts from the first place and the second address. The data corresponding to one of the two addresses is selected as the output data of the multiplexer, and is input to the multi-function array processor together with the first address; at the same time, the configuration file parser also performs the processing on the multi-function array processor. Calculate the configuration, calculate the output data of the multiplexer, and then store the calculation result in the first address; Further, the configuration file cache is a cache array dedicated to configuration environment files; the multi-function array processor is a reconfigurable multi-function array processor whose input is a 16-bit fixed-point number and can perform addition, multiplication, and sigmoid operations. The compound-granularity near-storage approximate acceleration structure of the long-short-term memory network according to claim 1, wherein the function configuration module bit width of the near-storage approximate acceleration operation is 16; The 0th to 7th bits of the function configuration module are the address configuration unit; among them, the 0th to the 2nd bits of the function configuration module are the address generator selection unit for selecting the address generator; the first bit of the function configuration module Bits 3 to 7 are the address selection unit, which is used to select the address in the address generator; The 8th to 11th bits of the function configuration module are the multiplexer configuration unit, which is used to select data for the multiplier; The 12th to 15th bits of the functional configuration module are the calculation configuration unit, which is used to indicate the type of operation. The calculation configuration unit can indicate addition, multiplication, logic operation, sigmoid operation, and approximate multiplication. The last calculation configuration unit Two bits are used to configure the number of iterations of approximate multiplication. The compound-granularity near-storage approximate acceleration method of the long and short-term memory network is characterized in that the steps of the acceleration method are as follows: Step S1, load the configuration file: load the configuration environment file into the configuration configuration file cache; Step S2, parsing the loaded configuration file, specifically includes the following three parallel steps: Step S2-1, resolve the address configuration: The configuration file parser performs the address configuration analysis operation, and the obtained address configuration files are loaded into the first address transmitter and the second address generator respectively, and used to select whether to take the address in the first address generator or select the second address to occur Address in the device; Further, the first address generator decides whether to select the corresponding first address from the first data storage module according to the address configuration file and fill it into the first address generator, and the second address generator decides whether to select from the second address according to the address configuration file. Select the corresponding second address in the data storage module and fill it into the second address generator; Step S2-2, parse the multiplexer configuration: The configuration file parser performs the multiplexer configuration analysis operation, obtains the multiplexer configuration file, and the multiplexer selects the data source; Further, the first address that is poured into the first address generator and the second address that is poured into the second address generator are both input into the multiplexer; After being configured, the multiplexer selects data corresponding to an address from the first address and the second address as the output data of the multiplexer, and is input into the multifunctional array processor together with the first address; Step S2-3, analyze and calculate the configuration: The configuration file parser performs the calculation configuration of the multi-function array processor. The reconfigurable multi-function array processor (RMPA) is calculated and configured to perform RMPA calculation on the output data of the multiplexer, and then the calculation result is stored in the first Address; In step S3, it is judged whether there are any more configuration files, if there are any more configuration files, return to step S1, and if there are no more configuration files, then it ends. The compound-granularity near-storage approximate acceleration structure of the long-short-term memory network according to claim 1, characterized in that: based on a compound-granularity task division strategy, stored in the first part and S _{0i of the} first data storage module, and stored in the second The second part of the data storage module and the _{calculation steps of S 1i} are as follows, where i represents the calculation step: Step A01, suppose that at the t-th time, the network reads the t-th input x _t , and configures the input gate i, the forget gate f, the memory unit c, and the output gate o response weight b and offset value W. The first part and , The second part and meet the following announcements:

Step A02: Calculate the response values of input gate i, forget gate f, memory unit c, and output gate o. The network reads the state value h _{t-1 of the} hidden layer at time t-1, and the first part and the second part are satisfied The following announcement:

Step A03: Calculate the response values of input gate i, forget gate f, memory cell c and output gate o, the network reads the memory cell vector value c _t-1 at time t-1, the first part and the second part satisfy the following Publicity:

Step A04: Calculate the response values of the input gate i, the forget gate f, the memory unit c, and the output gate o. The sum of the first part and the second part meet the following announcements:

Step A05: Calculate the vector values i _t and f _{t of the} input gate i and the forget gate f, and calculate the response values of the memory unit c and the output gate o. The sum of the first part and the second part meet the following announcements:

In the above _{formula, i t = σ (W ix} x t + W ih h t-1 + W ic c t-1 + b i), f t = σ (W fx x t + W fh h t-1 + W _fc c _t-1 +b _f ), where σ is the sigmoid function; Step A06: Calculate the response values of the memory unit c and the output gate o, and the first part sum and the second part sum meet the following announcements:

In step A07, the vector value c _{t of the} memory unit c is calculated, and the response value of the output gate o is calculated. The sum of the first part and the second part satisfy the following announcements:

In the above formula, c _t =f _t ⊙c _t-1 +i _t ⊙φ(W _cx x _t +W _ch h _t-1 +b _c ), where ⊙ represents the element-wise multiplication operation, and φ is the hyperbolic tangent function ； Step A08, calculate the response value of the output gate o, the first part and the second part meet the following announcements:

Step A09, calculate the response value of the output gate o, the first part and the second part meet the following announcements:

Step A10, calculate the vector value of the output gate o, the first part sum and the second part sum satisfy the following publicity:

In the above formula, o _t =σ(W _ox x _t +W _oh h _t-1 +W _oc c _t-1 +b _o ); Step A11, the state value h _{t of the} hidden layer at time t is calculated, and the first part sum and the second part sum satisfy the following publicity:

In the above formula, h _t =o _t ⊙φ(c _t ).

Images