WO2024216857A1 - Sparse spiking neural network accelerator based on ping-pong architecture - Google Patents

Abstract

The present application provides a sparse spiking neural network accelerator based on a ping-pong architecture. Compressed weight values are transmitted to a compressed weight calculation module, and a sparse spike detection module is used for extracting an effective spike index from a spike input signal, thereby preventing subsequent spike signals from all participating in an operation, and reducing the amount of calculation; and the compressed weight calculation module accumulates a non-zero value among the compressed weight values to a membrane potential of a neuron according to the effective spike index, and finally decides whether to emit a spike or not. Compared with a traditional technical solution that all synapses in a synaptic crossbar array are activated and participate in an operation, in the present application, only a synaptic weight corresponding to the effective spike index is activated, and other synapses do not participate in the operation, thereby reducing the amount of calculation, reducing the operating power consumption of the whole chip, and improving the operating speed, energy efficiency and area efficiency of a spiking neural network.

Description

Sparse spiking neural network accelerator based on ping-pong architecture

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese patent application No. 202310410779.3, filed on April 17, 2023, and entitled “Sparse Pulse Neural Network Accelerator Based on Ping-Pong Architecture”, which is incorporated herein by reference in its entirety.

Technical Field

The present application relates to the field of artificial intelligence technology, and in particular to a sparse pulse neural network accelerator based on a ping-pong architecture.

Background Art

Spiking Neural Network (SNN) is a computing model with great potential, which can improve the computing power of machines due to its low power consumption and high concurrency. It is considered to be the future of artificial intelligence research.

Since the signal conduction mechanism of neurons in spiking neural networks does not match the traditional von Neumann computer architecture, it is urgent to design a suitable hardware accelerator for spiking neural networks to run spiking neural networks. Current neuromorphic accelerators often use a synaptic cross array with a regular structure and fixed size to directly store the synaptic connection matrix of the SNN model. Regardless of the synaptic sparsity, all synapses will participate in the calculation, increasing the amount of calculation, which makes the spatial sparsity of the SNN model unable to be reflected on this neuromorphic accelerator. Another accelerator design scheme is to cache input pulses in the form of a bitmap, which will cause the hardware to judge the pulse validity of each bit in the input pulse vector, increasing the calculation time. As a result, the time sparsity of the pulse signal cannot be used to improve the hardware's calculation speed.

It can be seen that the current neural network accelerators cannot fully utilize the potential performance advantages of the SNN model, resulting in insufficient power consumption, insufficient computing speed, and insufficient energy efficiency for the operation of the SNN model.

Summary of the invention

The present application provides a sparse pulse neural network accelerator based on a ping-pong architecture to solve the problem that either all synapses are involved in the operation or the input Every symbol in the pulse signal must participate in the calculation, which leads to the defects of high chip power consumption and large amount of calculation, thus realizing the low power consumption and low latency operation of the pulse neural network.

The present application provides a sparse pulse neural network accelerator based on a ping-pong architecture, including a pulse input interface, a weight and neuron parameter input interface, a sparse pulse detection module, a compression weight calculation module, and a leakage integral issuance module; wherein,

The pulse input interface is used to receive a pulse input signal and input the pulse input signal to the sparse pulse detection module;

The weight and neuron parameter input interface is used to receive the compressed weight value and input the compressed weight value into the compressed weight calculation module;

The sparse pulse detection module is used to extract a valid pulse index from the pulse input signal; the valid pulse index is used to characterize the position of a non-zero value in the pulse input signal;

The compression weight calculation module is used to decompress the compression weight value according to the effective pulse index to obtain an effective weight matrix; calculate the weighted sum of the effective weight matrix and the pulse input signal to obtain the membrane potential increment on each neuron; and use the membrane potential increment on each neuron to update the membrane potential accumulation corresponding to each neuron;

The leaky integral issuing module is used to determine the magnitude relationship between the updated membrane potential accumulation amount and a preset threshold value, and determine the output pulse result corresponding to each neuron according to the magnitude relationship.

A sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application further includes a pulse buffer module group; the pulse buffer module group includes a first pulse buffer module and a second pulse buffer module;

The pulse buffer module group is used to control the read and write states of the first pulse buffer module and the second pulse buffer module in each buffer cycle in a ping-pong switching manner, so that one pulse buffer module is in a read state and the other pulse buffer module is in a write state in each buffer cycle.

A sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application also includes a weight cache module group; the weight cache module group includes a first weight cache module and a second weight cache module;

The weight cache module group is used to control the read and write status of the first weight cache module and the second weight cache module in each cache cycle in a ping-pong switching manner, so that one of the weight cache modules is in a read state and the other weight cache module is in a write state in each cache cycle.

According to a sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application, it also includes a neuron parameter cache module group; the neuron parameter cache module group includes a first neuron parameter cache module and a second neuron parameter cache module;

The neuron parameter cache module group is used to control the read and write states of the first neuron parameter cache module and the second neuron parameter cache module in each cache cycle in a ping-pong switching manner, so that one of the neuron parameter cache modules is in a read state and the other neuron parameter cache module is in a write state in each cache cycle.

According to a sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application, the sparse pulse detection module is further used to divide the pulse input sequence corresponding to the pulse input signal into a plurality of subsequences;

Performing a bitwise OR operation on each group of subsequences with itself in turn to obtain a bitwise OR operation result; if the bitwise OR operation result is all 0, then the operation of the current group of subsequences is terminated;

If the bitwise OR operation result is not all 0, the bitwise OR operation result is used as the current sequence to be detected, and multiple rounds of detection are performed on the current sequence to be detected. In each round of detection, the current sequence to be detected is subtracted by 1 to obtain a difference; the difference is bitwise ANDed with the current sequence to be detected to obtain a bitwise AND operation result; the bitwise AND operation result is bitwise XORed with the current sequence to be detected to obtain a valid pulse one-hot code; the valid pulse one-hot code is binary-converted to obtain a valid pulse index; it is determined whether the bitwise AND operation result is all 0. If so, the detection of the current sequence to be detected is terminated, and the step of performing a bitwise OR operation on each group of subsequences with itself in turn to obtain a bitwise OR operation result is returned; if not, the bitwise AND operation result is used as the current sequence to be detected, and the step of subtracting 1 from the current detection sequence to obtain a difference is returned.

According to a sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application, the compression weight calculation module includes a row offset calculation module, a column index weight calculation unit, a column index encoding module, a non-zero weight module, a weight distributor and a processing unit array; wherein,

The row offset calculation unit is used to obtain the row offset of the current row and the row offset of the next row adjacent to the current row;

The column index weight calculation unit is used to parse the row offset of the current row and the row offset of the next row adjacent to the current row to obtain the starting address and the ending address of the column index encoding module and the non-zero weight module; according to the starting address and the ending address, The index coding module and the non-zero weight module obtain non-zero weight and column index Delta coding;

The weight distributor comprises an adder chain composed of a preset number of adders, wherein the adder chain is used to use the non-zero weight and the Delta code as inputs of a processing unit array;

The processing unit array includes a preset number of processing units, each processing unit uses an adder to perform calculations, and the membrane potential accumulation amount is updated according to the calculation results of the adder.

According to a sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application, the column index weight calculation unit is also used to end the processing of the current row when the starting address exceeds the ending address.

According to a sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application, each processing unit also includes a weight mask generation module, a multiplexer and an adder;

The weight mask generating module is used to generate a weight mask according to the weight distribution state;

The processing unit is also used to use the weight mask and the non-zero weight as inputs of a multiplexer, so that the multiplexer obtains a valid non-zero value after filtering according to the weight mask, and uses the valid non-zero value and the membrane potential accumulation as inputs of an adder to obtain the updated membrane potential accumulation output by the adder.

According to a sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application, the leakage integral issuance module is further used to add the updated membrane potential accumulation amount to the preset leakage value to obtain a leakage integral value; if the leakage integral value is greater than the preset threshold, the output pulse result is determined to be a pulse signal; if the leakage integral value is less than or equal to the preset threshold, the pulse result is determined to be no pulse issuance.

A sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application also includes a pulse output interface;

The leakage integral issuance module is further used to send the updated membrane potential accumulation amount to the pulse output interface and reset the membrane potential accumulation amount when it is determined that the updated membrane potential accumulation amount is greater than the preset threshold.

The sparse pulse neural network accelerator based on the ping-pong architecture provided in this application transmits the compression weight value to the compression weight calculation module, and uses the sparse pulse detection module to extract the effective pulse index from the pulse input signal, thereby avoiding the subsequent participation of each pulse signal in the calculation, reducing the amount of calculation. The compression weight calculation module accumulates the non-zero value in the above compression weight value to the membrane potential of the neuron according to the effective pulse index, and finally decides whether to issue a pulse or not. Compared with the traditional Compared with the case where all synapses in the synaptic crossbar array are activated and participate in the calculation, in the present application, only the synaptic weights corresponding to the valid pulse index are activated, and other synapses do not participate in the calculation, thereby reducing the amount of calculation, reducing the operating power consumption of the entire chip, and improving the operating speed, energy efficiency and area efficiency of the pulse neural network.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic diagram of functional modules of a sparse pulse neural network accelerator based on a ping-pong architecture provided by the present application;

FIG2 is a schematic diagram of matrix elements of a row compression matrix provided by the present application;

FIG3 is a schematic diagram of the principle of LIF neurons provided by the present application;

FIG4 is a schematic diagram of a flow chart of a ping-pong operation method provided by the present application;

FIG5 is a schematic diagram of a hardware circuit of a sparse pulse detection module provided in the present application;

FIG6 is a schematic diagram of the structural principle of a compression weight calculation module provided by the present application;

FIG7 is a schematic diagram of the functional modules of the compression weight calculation module provided by the present application;

FIG8 is a schematic diagram of the hardware circuit structure of the column index weight calculation unit provided by the present application;

FIG9 is a schematic diagram of adder chain decoding provided by the present application;

FIG10 is a schematic diagram of the structure of the PE array provided in the present application.

DETAILED DESCRIPTION

In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

The specific implementation of the present application is described below in conjunction with Figures 1 to 10:

The sparse pulse neural network accelerator based on the ping-pong architecture provided in the embodiment of the present application has The functional module is shown in FIG1 , including a pulse input interface 101, a weight and neuron parameter input interface 102, a sparse pulse detection module 103, a compression weight calculation module 104, and a leakage integral issuance module 105; wherein,

The pulse input interface 101 is used to receive a pulse input signal and input the pulse input signal to the sparse pulse detection module 103;

Specifically, the pulse input signal is transmitted to the sparse pulse detection module 103 through the pulse input interface 101 .

The weight and neuron parameter input interface 102 is used to receive the compressed weight value and input the compressed weight value to the compressed weight calculation module 104;

Among them, the compressed weight value refers to the weight value in the sparse matrix storage format. Since there are many weight parameters in the pulse neural network model, using a sparse data storage format can save a lot of storage space and speed up the calculation. The sparse matrix representation format used in this application is CSR (Compressed Sparse Row, row compression) format.

Specifically, the compression weight value stored in the CSR format is input to the compression weight calculation module 104 through the weight and neuron parameter input interface 102 .

The sparse pulse detection module 103 is used to extract a valid pulse index from the pulse input signal; the valid pulse index is used to characterize the position of a non-zero value in the pulse input signal.

Since the pulse input signal is a sparse vector/tensor (the so-called sparse means having a large number of zeros), the above sparse vector/tensor needs to be multiplied by the synaptic weight during the calculation process, which means that most of the synaptic weights are multiplied by "0", and the result is still 0. In order to save computing power, this application does not need to read each bit in the pulse input signal from the memory, but only needs to read the valid pulse (i.e., the pulse signal with a non-zero value). In order to reduce the overall power consumption of the chip, this application only activates the neurons that need to be activated in the synaptic cross matrix, that is, activates the corresponding neurons in the synaptic cross array according to the position of the non-zero value in the pulse input signal. In order to determine which positions of neurons need to be activated, this application uses a sparse pulse detection module 103 to extract a valid pulse index from the above pulse input signal, wherein the valid pulse index is used to represent the position of the non-zero value in the pulse input signal.

The compression weight calculation module 104 is used to decompress the compression weight value received from the chip to obtain an effective weight matrix according to the effective pulse index; calculate the weighted sum of the effective weight matrix and the pulse input signal to obtain the membrane potential increment on each neuron; and use each neuron to calculate the effective weight matrix. The membrane potential increment on the neuron updates the accumulated membrane potential corresponding to each neuron.

Among them, the effective weight matrix is shown as the matrix on the left in Figure 2, and the effective weight matrix is a matrix obtained by restoring the compressed weight value according to the above-mentioned effective pulse index. The compressed weight value is shown as the "non-zero value" in Figure 2, and each element in the compressed weight value is a non-zero value in the effective weight matrix. Due to the limited on-chip storage space and the fact that neural networks often have a large number of parameters and weights, in order to save storage space, this application uses the CSR (Compressed Row Storage) format to store weights. The CSR storage format uses three one-dimensional arrays (including row offsets, column indexes, and non-zero values) to represent a sparse matrix (i.e., the effective weight matrix), where the row index array is used to store the cumulative count of non-zero values of the current row and the previous row. For example, the mth element in the row offset array represents the number of all non-zero values above the mth row in the effective weight matrix. For example, the second element "4" in the row offset array in Figure 2 represents the effective weight matrix The total number of non-zero values in all rows above the 2nd row (row numbers are encoded starting from 0) is 4 (1, 7, 2, 8 respectively), and so on. You only need to subtract two adjacent elements in the row offset array to know the non-zero values contained in each row of the effective weight matrix. For example, for the i-th row in the effective weight matrix, you only need to read the i+1th element value and the i-th element value in the row offset array, and subtract the two to calculate the number of non-zero values in the i-th row in the effective weight matrix; the column index array is used to store the column index of each element in the non-zero value array. For example, the column index corresponding to "7" in the non-zero value array is 1, indicating that 7 is located in the 1st column of the effective weight matrix (index is encoded starting from 0).

Specifically, the compressed weight calculation module 104 decompresses the compressed weight values stored in the CSR format to obtain an effective weight matrix; calculates the weighted sum of the effective weight matrix and the pulse input signal to obtain the membrane potential increment on each neuron; and uses the membrane potential increment on each neuron to accumulate the membrane potential accumulation corresponding to each neuron to obtain an updated membrane potential accumulation.

The leakage integral issuing module 105 is used to determine the magnitude relationship between the updated membrane potential accumulation amount and a preset threshold value, and determine the output pulse result corresponding to each neuron according to the magnitude relationship.

Specifically, as shown in FIG3 , FIG3 is a structure diagram of the spiking neural network algorithm used in this application. The neurons of the spiking neural network supported by this application are linear LIF (Leaky Integrate-and-Fire) models. After the accumulated pulse timing sequence is input, Performing linear leakage operation, threshold comparison and pulse firing, the membrane potential dynamics equation of the LIF neuron model is as follows:
V _j (t+1)＝V _j (t)+∑x _i w _ij -λ _j (1)

Where V _j (t) is the accumulated value of the membrane potential of neuron j at time t, w _ij is the i-th synaptic weight corresponding to neuron j, _xi is the pulse input signal value of the i-th synapse, and λ _j is the linear leakage of neuron j.

The leakage integral issuing module 105 determines the relationship between the updated membrane potential accumulation amount V _j (t+1) and the preset threshold value. If it is greater than the preset threshold value, a neural pulse signal is emitted and the membrane potential accumulation amount on the neuron is reset.

In the above embodiment, by transmitting the compression weight value to the compression weight calculation module, the sparse pulse detection module is used to extract the effective pulse index from the pulse input signal, thereby avoiding that each subsequent pulse signal is involved in the calculation, reducing the amount of calculation, and the compression weight calculation module adds the non-zero value in the above compression weight value to the membrane potential of the neuron according to the effective pulse index, and finally decides whether to emit a pulse or not. Compared with the traditional synaptic cross array in which all synapses are activated and participate in the calculation, in this application, only the synaptic weight corresponding to the effective pulse index is activated, thereby reducing the amount of calculation, reducing the operating power consumption of the entire chip, and improving the operating speed, energy efficiency and area efficiency of the pulse neural network.

In one embodiment, as shown in FIG1 , the sparse pulse neural network accelerator based on the ping-pong architecture further includes a pulse cache module group 106, and the pulse cache module group 106 includes a first pulse cache module and a second pulse cache module; wherein the pulse cache module group 106 is used to control the read and write states of the first pulse cache module and the second pulse cache module in each cache cycle in a ping-pong switching manner, so that in each cache cycle, one of the pulse cache modules is in a read state and the other pulse cache module is in a write state.

Specifically, the sparse pulse neural network accelerator based on the ping-pong architecture is internally provided with two pulse buffer modules RAM (Random Access Memory) for encoding the pulse input signal received from outside the chip to obtain the pulse input code suitable for calculation. During the encoding process, the entire system can update the other RAM while one RAM completes the calculation. Correspondingly, the output pulse code also needs to be decoded into the output pulse signal. During the decoding process, the ping-pong operation method is also adopted. While one RAM completes the calculation, the other RAM is updated.

In the above-mentioned embodiment, the pulse buffer module group is used to implement ping-pong buffering in the process of encoding pulse input signals or decoding pulse output signals, thereby accelerating simultaneous I/O operations and data processing operations and improving the throughput of the accelerator.

In one embodiment, as shown in FIG1 , the sparse pulse neural network accelerator based on the ping-pong architecture further includes a weight cache module group 107, which includes a first weight cache module and a second weight cache module; wherein the weight cache module group 107 is used to control the read and write states of the first weight cache module and the second weight cache module in each cache cycle in a ping-pong switching manner, so that one of the weight cache modules is in a read state and the other weight cache module is in a write state in each cache cycle.

Specifically, the sparse pulse neural network accelerator based on the ping-pong architecture is internally provided with two weight cache module RAMs, which are used to decompress the compressed weight values in the CSR format received from outside the chip to obtain the effective weight matrix. During the decompression process, the entire system can update the other RAM while one RAM completes the decompression.

The above-mentioned embodiment utilizes the weight cache module group to implement ping-pong cache in the process of decompressing the compressed weight values in the CSR format, accelerates the simultaneous I/O operations and data processing operations, and improves the throughput of the accelerator.

In one embodiment, the accelerator further includes a neuron parameter cache module group 108; the neuron parameter cache module group 108 includes a first neuron parameter cache module and a second neuron parameter cache module; the neuron parameter cache module group 108 is used to control the read and write states of the first neuron parameter cache module and the second neuron parameter cache module in each cache cycle in a ping-pong switching manner, so that in each cache cycle, one of the neuron parameter cache modules is in a read state and the other neuron parameter cache module is in a write state.

Specifically, the sparse pulse neural network accelerator based on the ping-pong architecture is internally provided with two neuron parameter cache module RAMs, which are used to decode the neuron parameters received from outside the chip to obtain neuron parameters suitable for calculation. During the decoding process, the entire system can update the other RAM while one RAM is completing decompression.

Furthermore, the ping-pong operation algorithm used in the present application includes three dimensions of control, namely, how many time slots the input pulse of each time step contains, how many time steps each group of neurons needs to calculate, and how many groups each layer of the network needs to be divided into for batch calculation.

As shown in Figure 4, a two-layer 1024-512-256 fully connected spiking neural network is used as For example, the accelerator will receive data from the pulse RAM#0, weight RAM#0, and neuron parameter RAM#0 from the outside after power-on. After receiving the first global synchronization instruction "Sync_all", the accelerator's weight RAM#1 and neuron parameter RAM#1 receive data from the outside, and the data of pulse RAM#0, weight RAM#0, and neuron parameter RAM#0 are sent to the core computing unit, and the calculation results are sent to pulse RAM#1. At this time, the accelerator is calculating the first 256 neurons of the first layer. After receiving the second global synchronization instruction "Sync_all", the accelerator's weight RAM#0 and neuron parameter RAM#0 receive data from the outside, and the data of pulse RAM#0, weight RAM#1, and neuron parameter RAM#1 are sent to the core computing unit, and the calculation results are sent to pulse RAM#1. At this time, the accelerator is calculating the last 256 neurons of the first layer. After receiving the third global synchronization instruction "Sync_all", the accelerator's weight RAM#1 and neuron parameter RAM#1 receive data from the outside, and the data of pulse RAM#1, weight RAM#0 and neuron parameter RAM#0 are sent to the core computing unit. The calculation results are sent to pulse RAM#0 and the outside of the accelerator. At this time, the accelerator is calculating the 256 neurons in the second layer, and the entire calculation process is completed.

In the above-mentioned embodiment, the neuron parameter cache module group is used to implement ping-pong cache of neuron parameters during the decoding process, thereby accelerating simultaneous I/O operations and data processing operations and improving the throughput of the accelerator.

In one embodiment, the sparse pulse detection module 105 is further used to divide the pulse input sequence corresponding to the pulse input signal into a plurality of subsequences;

Performing a bitwise OR operation on each group of subsequences with itself in turn to obtain a bitwise OR operation result; if the bitwise OR operation result is all 0, then the operation of the current group of subsequences is terminated;

If the bitwise OR operation result is not all 0, the bitwise OR operation result is used as the current sequence to be detected, and multiple rounds of detection are performed on the current sequence to be detected. In each round of detection, the current sequence to be detected is subtracted by 1 to obtain a difference; the difference is bitwise ANDed with the current sequence to be detected to obtain a bitwise AND operation result; the bitwise AND operation result is bitwise XORed with the current sequence to be detected to obtain a valid pulse one-hot code; the valid pulse one-hot code is binary-converted to obtain a valid pulse index; it is determined whether the bitwise AND operation result is all 0. If so, the detection of the current sequence to be detected is terminated, and the step of performing a bitwise OR operation on each group of subsequences with itself in turn to obtain a bitwise OR operation result is returned; if not, the bitwise AND operation result is used as the current sequence to be detected, and the step of subtracting 1 from the current detection sequence is returned. Then get the difference step.

The pulse input sequence is a sequence obtained by encoding the pulse input signal. The valid pulse index refers to the position of the non-zero value in the pulse input sequence.

Specifically, as shown in FIG5 , FIG5 shows the internal circuit structure diagram of the sparse pulse detection module 105, including logic gate circuits such as an OR gate, a multiplexer, a D flip-flop, an adder, an AND gate, and an XOR gate. The sparse pulse detection module 105 is mainly responsible for extracting the effective pulse index of the input pulse sequence. In order to reduce the critical path delay, the present application groups the input pulse sequence. For example, a 1024-bit input pulse sequence is divided into 16 groups of 64-bit subsequences. Each group of 64-bit subsequences is first bitwise ORed with itself to obtain a bitwise OR operation result. If the bitwise OR operation result is all 0, it means that all 64 bits in this group of subsequences are 0, then the operation of this group of subsequences is terminated, that is, the calculation of the 64-bit subsequence is skipped.

If the result of the bitwise OR operation is not all 0, it is necessary to further detect the position of the non-zero value in the current group of subsequences. The detection method specifically includes: taking the above-mentioned bitwise OR operation result as the current sequence to be detected, performing multiple rounds of detection on the current sequence to be detected, in each round of detection, first subtracting 1 from the current sequence to be detected (for example, the above-mentioned 64-bit subsequence) to obtain a difference, and then performing a bitwise AND operation on the difference and the current sequence to be detected itself to obtain a bitwise AND operation result; performing a bitwise XOR operation on the bitwise AND operation result and the data before the above-mentioned subtraction of 1 (that is, the current sequence to be detected itself) to obtain a valid pulse unique hot code; at the same time, judging whether the above-mentioned bitwise AND operation result is all 0, if so, ending the detection of this group of subsequences and entering the detection of the next group of subsequences, in the example of the present embodiment, that is, entering the detection of the next group of 64-bit subsequences; if not, taking the bitwise AND operation result as the current sequence to be detected, and returning to the step of "subtracting 1 from the current sequence to be detected and obtaining a difference". This cycle is repeated until each subsequence is detected, and a number of valid pulse one-hot codes can be obtained. Through a one-hot code to binary code decoding unit, a valid pulse index in binary code form corresponding to the pulse input signal can be obtained.

In the above embodiment, the sparse pulse detection module 105 is obtained by combining multiple logic gate circuits, which can realize effective index detection of sparse pulses.

In one embodiment, as shown in FIG. 6, FIG. 6 shows a structural schematic diagram of the compression weight calculation module 104, which includes a row offset module (Row Offset Module, ROM), a column index Delta coding module (Column Delta Module, CDM), a nonzero weight value module (Nonzero Weight Value Module, NWVM), and a PE (Processing Element, processing unit) array, in which each PE unit is used to complete the calculation of membrane potential increment, that is, the dot product of the pulse input signal and the synaptic weight, that is, x _i w _ij in formula (1). In Figure 6, a 1024×256 synaptic crossbar array is used as an example for explanation. 1024 represents 1024 fan-in (i.e., 1024 axons), and 256 represents 256 hardware neurons, that is, each hardware neuron has 1024 fan-in, but not every fan-in has a valid pulse signal. The valid pulse index transmitted from the sparse pulse detection module will activate the row corresponding to the valid pulse index, and then the row offset module (ROM) calculates the corresponding row offset, the column index Delta encoding module (CDM) reads the column index of the non-zero value of the row according to the row offset, and the non-zero weight module (NWVM) accumulates the corresponding non-zero value to the membrane potential according to the column index. When all valid pulses in the input pulse sequence are processed, all membrane potential accumulation values will be sent to the leaky integral release neuron dynamics behavior module for LIF calculation.

Since the effective weight matrix is represented by three parameters, namely, row offset, column index and non-zero value, as shown in Figure 2. Accordingly, in order to restore the effective weight matrix according to the above three parameters, as shown in Figure 7, the storage of the above three parameters in the accelerator is respectively set: row offset storage module (Row Offset Module, ROM, which is further divided into Even ROM and Odd ROM), column index Delta encoding storage module (Column Delta Module, CDM) and non-zero value storage module (NWVM). For the pipeline, the row offset storage module is further divided into even row offset storage module (Even ROM) and odd row offset storage module (Odd ROM). The pipeline diagram of the compression weight calculation process is shown in FIG7. For the input pulse pointing to the i-th row, the CSR decoder reads the row offset of the i-th row and the row offset of the i+1-th row from the Even ROM (even row offset module) and the Odd ROM (odd row offset module) respectively. The number of non-zero values corresponding to the pulse of the i-th row can be obtained by subtracting the two, thereby obtaining the memory access address and the number of memory access rounds for the column index Delta coding storage module (CDM) and the non-zero value storage module (NWVM). Since only one row offset memory access and calculation is required for the pulse of the i-th row, but more than one column index and weight memory access and calculation may be required for the i-th row, which will lead to a mismatch in the throughput of the two types of data structure processing pipelines. Therefore, in order to avoid inefficient uneven pipeline pauses, the CSR decoder is decoupled into a row offset calculation unit and a column index weight calculation unit in the circuit design, and the two are synchronized using a FIFO (First Input First Output) memory, as shown in FIG7.

The circuit structure of the column index weight calculation unit is shown in FIG8 . After parsing the obtained row offset RO _i of the i-th row and the row offset RO _i+1 of the i+1th row, the access start address Addr _i and the end address Addr _i+1 of the column index Delta coding storage module (CDM) and the non-zero value storage module (NWVM) are obtained respectively. When the output signal of the MAE (Masked Autoencoder) is valid, the column index ΔCI _j and the non-zero weight (WV) in the Delta coding format are read from the column index Delta coding storage module (CDM) and the non-zero value storage module (NWVM) according to Addr _i in each clock cycle and sent to the subsequent circuit unit (i.e., the corresponding PE unit). When Addr _i exceeds Addr _i+1 , the column index weight calculation unit will end the processing of the i-th row and obtain the row offset of a new row from the row offset storage module.

As shown in FIG9 , the column index (i.e., ΔCI _j ) of the Delta code read out from the column index Delta code storage module (CDM) in each clock cycle is decoded into the column index (i.e., CI _j ) through the adder chain. Since the ΔCI (i.e., column index Delta code) that does not belong to the current row may be read out from the column index Delta code module (CDM) in each clock cycle, the ΔCI that does not belong to the current row is filtered out through the MUX (multiplexer) to obtain the filtered Delta code ΔCI ^′ . If the current clock cycle is the first cycle of the row operation, ΔCI′ ₀ will be used as CI ₀ of this cycle; if the current clock cycle is not the first cycle of the row operation, ΔCI′ ₀ will be added to CI ₃₁ of the previous cycle. The other CI _j = CI _j-1 + ΔCI′ _j .

The structure of the PE array is shown in Figure 10, which shows a PE array containing 32 PE units; each PE unit contains a 16-bit adder and 2 MUXs (multiple selectors), and the accumulated values of the membrane potentials of all dendrites are input to the MUX1 (multiple selector of multiple selection 1) of each PE. After obtaining 32 pairs of CI _j (column index) and WV (non-zero weight), the weight distributor uses each pair of CI _j (jth column index) and WV _j (jth non-zero weight) as the input of the corresponding PE _j : Among them, CI _j will be used as the multiple selection signal of MUX1, and the corresponding (The accumulated value of the membrane potential of the neuron corresponding to the jth column index) is used as an operand of the adder; WV _j will be input to one of the two input terminals of MUX2. After filtering by the weight mask, the effective WV _j will be used as the other operand of the adder. The weight distributor is updated according to the result calculated by the adder. The updated as the cumulative value for the next cycle.

The role of the weight mask is to filter out invalid non-zero weights WV to avoid calculation errors. There are two reasons why invalid non-zero weights WV exist in the calculation: (1) When accessing NWVM, the weights at the same memory address of NWVM but not belonging to the row will also be read out as invalid. Zero weight WV; (2) In order to realize the fan-in expansion function, the 128 columns of the weight matrix are divided into multiple shared fan-in dendrite clusters. When accessing NWVM, the weights of other dendrites that do not belong to the shared fan-in dendrite cluster receiving the current input pulse vector will also be read out as non-zero weight WV.

In the above embodiment, the processing level of the pipeline is further improved by providing an odd-numbered row offset module and an even-numbered row offset module.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims (10)

A sparse pulse neural network accelerator based on a ping-pong architecture includes a pulse input interface, a weight and neuron parameter input interface, a sparse pulse detection module, a compression weight calculation module, and a leakage integral issuance module; wherein, The pulse input interface is used to receive a pulse input signal and input the pulse input signal to the sparse pulse detection module; The weight and neuron parameter input interface is used to receive the compressed weight value and input the compressed weight value into the compressed weight calculation module; The sparse pulse detection module is used to extract a valid pulse index from the pulse input signal; the valid pulse index is used to characterize the position of a non-zero value in the pulse input signal; The compression weight calculation module is used to decompress the compression weight value according to the effective pulse index to obtain an effective weight matrix; calculate the weighted sum of the effective weight matrix and the pulse input signal to obtain the membrane potential increment on each neuron; and use the membrane potential increment on each neuron to update the membrane potential accumulation corresponding to each neuron; The leaky integral issuing module is used to determine the magnitude relationship between the updated membrane potential accumulation amount and a preset threshold value, and determine the output pulse result corresponding to each neuron according to the magnitude relationship. The sparse pulse neural network accelerator based on the ping-pong architecture according to claim 1 further comprises a pulse buffer module group; the pulse buffer module group comprises a first pulse buffer module and a second pulse buffer module; The pulse buffer module group is used to control the read and write states of the first pulse buffer module and the second pulse buffer module in each buffer cycle in a ping-pong switching manner, so that one pulse buffer module is in a read state and the other pulse buffer module is in a write state in each buffer cycle. The sparse pulse neural network accelerator based on the ping-pong architecture according to claim 2 further comprises a weight cache module group; the weight cache module group comprises a first weight cache module and a second weight cache module; The weight cache module group is used to control the read and write states of the first weight cache module and the second weight cache module in each cache cycle in a ping-pong switching manner, so that one of the weight cache modules is in a read state and the other weight cache module is in a write state in each cache cycle. state. The sparse pulse neural network accelerator based on the ping-pong architecture according to claim 3 further comprises a neuron parameter cache module group; the neuron parameter cache module group comprises a first neuron parameter cache module and a second neuron parameter cache module; The neuron parameter cache module group is used to control the read and write states of the first neuron parameter cache module and the second neuron parameter cache module in each cache cycle in a ping-pong switching manner, so that one of the neuron parameter cache modules is in a read state and the other neuron parameter cache module is in a write state in each cache cycle. The sparse pulse neural network accelerator based on ping-pong architecture according to claim 1, wherein the sparse pulse detection module is further used to divide the pulse input sequence corresponding to the pulse input signal into a plurality of subsequences; Performing a bitwise OR operation on each group of subsequences with itself in turn to obtain a bitwise OR operation result; if the bitwise OR operation result is all 0, then the operation of the current group of subsequences is terminated; If the bitwise OR operation result is not all 0, the bitwise OR operation result is used as the current sequence to be detected, and multiple rounds of detection are performed on the current sequence to be detected. In each round of detection, the current sequence to be detected is subtracted by 1 to obtain a difference; the difference is bitwise ANDed with the current sequence to be detected to obtain a bitwise AND operation result; the bitwise AND operation result is bitwise XORed with the current sequence to be detected to obtain a valid pulse one-hot code; the valid pulse one-hot code is binary-converted to obtain a valid pulse index; it is determined whether the bitwise AND operation result is all 0. If so, the detection of the current sequence to be detected is terminated, and the step of performing a bitwise OR operation on each group of subsequences with itself in turn to obtain a bitwise OR operation result is returned; if not, the bitwise AND operation result is used as the current sequence to be detected, and the step of subtracting 1 from the current detection sequence to obtain a difference is returned. According to the sparse pulse neural network accelerator based on the ping-pong architecture of claim 1, wherein the compression weight calculation module includes a row offset operation unit, a column index weight operation unit, a row offset storage module, a column index Delta encoding storage module, a non-zero weight storage module, a weight distributor and a processing unit array; wherein, The row offset calculation unit is used to read the row offset of the current row and the row offset of the next row adjacent to the current row from the row offset storage module; The column index weight calculation unit is used to analyze the row offset of the current row and the The row offset of the next row adjacent to the current row obtains the starting address and the ending address of the column index encoding module and the non-zero weight module; according to the starting address and the ending address, the column index Delta code and the non-zero weight are obtained from the column index encoding module and the non-zero weight module respectively; The weight distributor comprises an adder chain composed of a preset number of adders, wherein the adder chain is used to use the non-zero weight and the Delta code as inputs of a processing unit array; The processing unit array includes a preset number of processing units, each processing unit uses an adder to perform calculations, and the membrane potential accumulation amount is updated according to the calculation results of the adder. According to the ping-pong architecture-based sparse pulse neural network accelerator of claim 6, wherein the column index weight calculation unit is further used to end the processing of the current row when the starting address exceeds the ending address. According to the ping-pong architecture-based sparse pulse neural network accelerator of claim 6, each processing unit further includes a weight mask generation module, a multiplexer and an adder; The weight mask generating module is used to generate a weight mask according to the weight distribution state; The processing unit is also used to use the weight mask and the non-zero weight as inputs of a multiplexer, so that the multiplexer obtains a valid non-zero value after filtering according to the weight mask, and uses the valid non-zero value and the membrane potential accumulation as inputs of an adder to obtain the updated membrane potential accumulation output by the adder. According to the sparse pulse neural network accelerator based on the ping-pong architecture of claim 1, wherein the leakage integral issuance module is further used to add the updated membrane potential accumulation amount to the preset leakage value to obtain a leakage integral value; if the leakage integral value is greater than the preset threshold, the output pulse result is determined to be a pulse signal; if the leakage integral value is less than or equal to the preset threshold, the pulse result is determined to be no pulse issuance. The sparse pulse neural network accelerator based on the ping-pong architecture according to claim 1 further comprises a pulse output interface; The leakage integral issuance module is further used to send the updated membrane potential accumulation amount to the pulse output interface and reset the membrane potential accumulation amount when it is determined that the updated membrane potential accumulation amount is greater than the preset threshold.