CN117391175A - A spiking neural network quantification method and system for brain-like computing platforms - Google Patents

Abstract

The invention discloses a pulse neural network quantification method and a pulse neural network quantification system for a brain-like computing platform, wherein the method comprises the following steps: loading and storing a pulse neural network model of the pre-training full-precision float32, and marking the pulse neural network model as a pulse neural network model T1; modifying the impulse neural network model T1 to obtain an impulse neural network model T2; training the pulse neural network model T2 in a quantization process by selecting a calibration data set to select a proper scaling quantization scale factor scale and a zero zp so as to minimize precision loss before and after quantization; deploying the trained impulse neural network model T2 to a brain-like computing platform; the truly asynchronous impulse neural network calculation is realized, and no precision is lost.

Description

A spiking neural network quantification method and system for brain-like computing platforms

Technical field

The invention belongs to the field of artificial intelligence model compression, and specifically relates to a pulse neural network quantification method and system.

Background technique

Impulsive neural network is the third generation neural network. Its principle is to simulate the impulse firing mode of biological neurons to realize information interaction between network layers. Due to its impulse characteristics, convolution and fully connected multiplication and addition operations are directly converted into addition operations, which can Significantly reduces the amount of calculations. Since both current CPUs and GPUs are synchronous computing architectures, it is difficult to take advantage of the spiking neural network.

Wentian is a massively parallel, general-purpose super brain-like computer based on spiking neural networks. Wentian is based on the computing architecture of ARM chips. The int8 quantized model can not only play a role in acceleration, but also combine with the hardware int8 architecture to achieve precision and lossless migration deployment. However, due to the sparsity of the spiking neural network, the loss of model accuracy has a serious impact on the overall accuracy of the model.

Contents of the invention

The present invention provides a pulse neural network quantification method for a brain-like computing platform to solve the technical problem of accuracy loss caused by insufficient representation accuracy.

In order to achieve the above objects, the technical solutions adopted by the present invention are:

The first aspect of the present invention provides a spiking neural network quantification method for a brain-like computing platform, including:

Load and save the pre-trained full-precision float32 spiking neural network model, recorded as spiking neural network model T1;

The process of modifying the spiking neural network model T1 to obtain the spiking neural network model T2 is:

Pseudo quantization nodes are inserted into the input layer and output layer of the spiking neural network model T1. The input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer inversely quantizes the output features from the int8 format to the full-precision float32 format;

Define the linear operator in the impulse neural network model T1 to the pseudo quantization node, dequantize the characteristic parameters of the input pseudo quantization node from the int8 format to the full precision float32 format, and then quantize it back to the int8 format after calculation by the pseudo quantization node;

Change the spiking neural network model T1 to the Euler numerical solution form, then fold the convolution operator, batch normalization process, and LIF neuron operator, and adjust the weights and activations of each layer in the spiking neural network model T1 The function inserts an observation node, which is used to calculate and save the maximum and minimum values of each group of weights under each channel, as well as the quantization range, to obtain the impulse neural network model T2;

Select the calibration data set to train the spiking neural network model T2 in the quantification process to select the appropriate scaling factor scale and zero point zp to minimize the loss of accuracy before and after quantification; deploy the trained spiking neural network model T2 to the brain-inspired computing platform .

Furthermore, the expression formula of changing the impulse neural network model T1 into Euler's numerical solution form is:

Among them, V(t) is the membrane potential at time t, V _reset is the reset potential, V _threshold is the threshold potential, X(t) is the input, S(t) is the pulse at time t, Φ is the step function, τ is the time decay constant.

Furthermore, methods for operator folding of convolution operators, batch normalization processing, and LIF neuron operators include:

The expression formulas of the convolution operator, batch normalization process, and LIF neuron operator of the spiking neural network model T1 are:

The expression formula for operator folding of the convolution operator, batch normalization processing, and LIF neuron operator is:

In the formula, w ⁿ represents the n-th layer weight, w ⁿ⁺¹ represents the n+1-th layer weight, b ⁿ⁺¹ represents the n+1-th layer bias, x ⁿ⁺² (t) represents the n+2-th layer t The output value at time, x ⁿ⁺¹ (t) represents the output value of the n+1th layer at time t, x ^n-1 (t) represents the output value of the n-1th layer at time t, x ⁿ (t) Represents the output value of the nth layer at the tth moment, σ ² is the variance, μ is the mean, ∈ = 1e-5;.

Furthermore, the calibration data set is selected to train the impulse neural network model T2 in the quantization process, so as to select the appropriate scaling and quantization amplification factor scale and zero point zp to minimize the loss of accuracy before and after quantization. Methods include:

Use the calibration data set to train the spiking neural network model T2, set the step function Φ in the back propagation process of the spiking neural network model T2 as a sigmoid function, and calculate and save the maximum value of each group of weights under each channel by observing the nodes. and the minimum value and quantization range, and then calculate the scaling quantization amplification factor scale and zero point zp; according to the scaling quantization amplification factor scale and zero point zp, perform the quantization operation from the full-precision float32 format to the int8 format or convert from the int8 format to the full-precision float32 format dequantization operation.

Furthermore, the quantization operation is performed from the full-precision float32 format to the int8 format. The expression formula is:

In the formula, x _Q is the quantized output feature, x represents the quantized input feature, and clamp(·) passes the quantified input feature x through Quantized truncated 0 to N _levels -1 range; N _levels are expressed as the maximum value in int8 format.

Further, perform an inverse quantization operation from int8 format to full-precision float32 format. The expression formula is:

x _float =(x _Q -zp)scale

In the formula, x _float represents the inverse quantization output feature; x _Q represents the quantization output feature.

Further, the method of deploying the trained spiking neural network model T2 to the brain-like computing platform includes: by creating a population according to the trained spiking neural network model T2 one-to-one, deploying the trained spiking neural network model T2 to On the brain-like computing platform, connect the folded weights of the operators to the neurons in the manner of specifying neuron numbers one-to-one with FromListConnector and assign the folded weight w ^n+1′ to the connection; set the bias current I_offset of the neuron. is b ^n+1′ , and the neuron membrane capacitance C is set to τ.

In a second aspect, the present invention provides a spiking neural network quantification system for a brain-like computing platform, including:

The loading module is used to load and save the pre-trained full-precision float32 impulse neural network model, recorded as impulse neural network model T1;

The model modification module is used to insert pseudo quantization nodes into the input layer and output layer of the impulse neural network model T1. The input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer dequantizes the output features from the int8 format. It is a full-precision float32 format; define the linear operator in the impulse neural network model T1 to the pseudo-quantization node, dequantize the characteristic parameters of the input pseudo-quantization node from the int8 format to the full-precision float32 format, and then quantize it back to int8 after calculation by the pseudo-quantization node Format; change the spiking neural network model T1 to the Euler numerical solution form, then fold the convolution operator, batch normalization process, and LIF neuron operator, and adjust the weight of each layer in the spiking neural network model T1 And the activation function is inserted into the observation node, which is used to calculate and save the maximum and minimum values of each group of weights under each channel and the quantization range, and obtain the spiking neural network model T2;

The training module is used to select the calibration data set to train the impulse neural network model T2 in the quantization process, so as to select the appropriate scaling factor scale and zero point zp to minimize the loss of accuracy before and after quantization;

The execution module is used to deploy the trained spiking neural network model T2 to the brain-like computing platform.

Further, the training module is used to select the calibration data set to train the impulse neural network model T2 in the quantization process, so as to select the appropriate scaling factor scale and zero point zp to minimize the loss of accuracy before and after quantization. Methods include:

Use the calibration data set to train the impulse neural network model T2, calculate and save the maximum and minimum values of each group of weights under each channel and the quantization range by observing the nodes, and then calculate the scaling and quantization amplification factor scale and zero point zp; according to the scaling The quantization factor scale and zero point zp perform a quantization operation from full-precision float32 format to int8 format or an inverse quantization operation from int8 format to full-precision float32 format.

In a third aspect, the present invention provides an electronic device, including a storage medium and a processor; the storage medium is used to store instructions; and the processor is used to operate according to the instructions to perform the method described in the first aspect.

Compared with the existing technology, the beneficial effects of the present invention are:

The present invention selects a calibration data set to train the spiking neural network model T2 in the quantification process, so as to select the appropriate scaling factor scale and zero point zp to minimize the loss of accuracy before and after quantification; and deploys the trained spiking neural network model T2 to the brain. Computing platform to achieve truly asynchronous spiking neural network calculations without loss of accuracy.

The present invention's convolution or fully connected multiplication and addition operations, batch normalization's multiplication and addition operations, and LIF neuron's multiplication and addition operations are all linear multiplication and addition calculations, so the three operators can be directly folded into one layer. , to avoid the accuracy loss caused by repeated quantization and anti-quantization operations during the quantization process; at the same time, because the three layers are folded into one layer, the network inference speed is greatly shortened.

Description of the drawings

Figure 1 is a flow chart of a spiking neural network quantification method for a brain-inspired computing platform provided by an embodiment.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to more clearly illustrate the technical solutions of the present invention, but cannot be used to limit the scope of the present invention.

Example 1

As shown in Figure 1, a spiking neural network quantification method for brain-like computing platforms includes:

Load and save the pre-trained full-precision float32 spiking neural network model, recorded as spiking neural network model T1;

The process of modifying the spiking neural network model T1 to obtain the spiking neural network model T2 is:

Pseudo quantization nodes are inserted into the input layer and output layer of the spiking neural network model T1. The input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer inversely quantizes the output features from the int8 format to the full-precision float32 format;

Define the linear operator in the impulse neural network model T1 to the pseudo quantization node, dequantize the characteristic parameters of the input pseudo quantization node from the int8 format to the full precision float32 format, and then quantize it back to the int8 format after calculation by the pseudo quantization node;

The expression formula of changing the impulsive neural network model T1 into Euler's numerical solution form is:

Among them, V(t) is the membrane potential at time t, V _reset is the reset potential, V _threshold is the threshold potential, X(t) is the input, S(t) is the pulse at time t, Φ is the step function, τ is the time decay constant.

Methods for operator folding of convolution operators, batch normalization processing, and LIF neuron operators include:

The expression formulas of the convolution operator, batch normalization process, and LIF neuron operator of the spiking neural network model T1 are:

The expression formula for operator folding of the convolution operator, batch normalization processing, and LIF neuron operator is:

In the formula, w ⁿ represents the n-th layer weight, w ⁿ⁺¹ represents the n+1-th layer weight, b ⁿ⁺¹ represents the n+1-th layer bias, x ⁿ⁺² (t) represents the n+2-th layer t The output value at time, x ⁿ⁺¹ (t) represents the output value of the n+1th layer at time t, x ^n-1 (t) represents the output value of the n-1th layer at time t, x ⁿ (t) Represents the output value of the nth layer at time t, σ ² is the variance, μ is the mean, ∈ = 1e-5; to avoid the accuracy loss caused by repeated quantization and anti-quantization operations during the quantization process; at the same time, due to the three layers Folded into one layer, the network inference speed is greatly shortened.

And insert observation nodes into the weights and activation functions of each layer in the spiking neural network model T1. The observation nodes are used to calculate and save the maximum and minimum values of each group of weights and the quantization range under each channel to obtain the spiking neural network model T2. ;

Select the calibration data set to perform gradient training of the quantization process on the impulse neural network model T2 to select the appropriate scaling and quantization amplification factor scale and zero point zp to minimize the loss of accuracy before and after quantization. Methods include:

Use the calibration data set to train the spiking neural network model T2, set the step function Φ in the back propagation process of the spiking neural network model T2 as a sigmoid function, and calculate and save the maximum value of each group of weights under each channel by observing the nodes. and the minimum value and quantization range, and then calculate the scaling quantization amplification factor scale and zero point zp; according to the scaling quantization amplification factor scale and zero point zp, perform the quantization operation from the full-precision float32 format to the int8 format or convert from the int8 format to the full-precision float32 format dequantization operation.

Perform quantization operation from full-precision float32 format to int8 format. The expression formula is:

In the formula, x _Q is the quantized output feature, x represents the quantized input feature, and clamp(·) passes the quantified input feature x through Quantized truncated 0 to N _levels -1 range; N _levels are expressed as the maximum value in int8 format.

Perform an inverse quantization operation from int8 format to full-precision float32 format. The expression formula is:

x _float =(x _Q -zp)scale

In the formula, x _float represents the inverse quantization output feature.

The method of deploying the trained spiking neural network model T2 to the brain-like computing platform includes: creating a population according to the trained spiking neural network model T2 one-to-one, and deploying the trained spiking neural network model T2 to the Wentian class On the brain computing platform, the Wentian brain-inspired computing platform is a massively parallel, general-purpose super brain-inspired computer based on spiking neural networks. The Wentian brain-inspired computing platform is based on the computing architecture of ARM chips. The int8 quantized model can not only play an acceleration role, but also combines with the hardware int8 architecture to achieve precision and lossless migration deployment. The architecture of the Wentian brain-inspired computing platform is consistent with Pulse. The working principle of the neural network is very gentle. When no pulse signal is received, the chip maintains a resting state, and calculation occurs only at the location of the neuron that received the pulse, which greatly reduces the energy consumption of calculation.

Connect the folded weight of the operator to the neurons in a way that specifies the neuron number one-to-one with FromListConnector, and assign the folded weight w ^n+1′ to the connection; set the bias current I_offset of the neuron to b ^n+1′ , and set the neuron membrane capacitance C to τ; achieving truly asynchronous pulse neural network calculations without loss of accuracy.

Example 2

This embodiment provides a spiking neural network quantification system for a brain-like computing platform. The spiking neural network quantification system described in this embodiment can be applied to the spiking neural network quantification method described in Embodiment 1. The spiking neural network quantification system include:

The loading module is used to load and save the pre-trained full-precision float32 impulse neural network model, recorded as impulse neural network model T1;

The model modification module is used to insert pseudo quantization nodes into the input layer and output layer of the impulse neural network model T1. The input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer dequantizes the output features from the int8 format. It is a full-precision float32 format; define the linear operator in the impulse neural network model T1 to the pseudo-quantization node, dequantize the characteristic parameters of the input pseudo-quantization node from the int8 format to the full-precision float32 format, and then quantize it back to int8 after calculation by the pseudo-quantization node Format; change the spiking neural network model T1 to the Euler numerical solution form, then fold the convolution operator, batch normalization process, and LIF neuron operator, and adjust the weight of each layer in the spiking neural network model T1 And the activation function is inserted into the observation node, which is used to calculate and save the maximum and minimum values of each group of weights under each channel and the quantization range, and obtain the spiking neural network model T2;

The training module is used to select the calibration data set to train the impulse neural network model T2 in the quantization process, so as to select the appropriate scaling factor scale and zero point zp to minimize the loss of accuracy before and after quantization;

The execution module is used to deploy the trained spiking neural network model T2 to the brain-like computing platform.

The training module is used to select the calibration data set to train the impulse neural network model T2 in the quantization process, so as to select the appropriate scaling factor scale and zero point zp to minimize the loss of accuracy before and after quantization. The method includes:

Use the calibration data set to train the impulse neural network model T2, calculate and save the maximum and minimum values of each group of weights under each channel and the quantization range by observing the nodes, and then calculate the scaling and quantization amplification factor scale and zero point zp; according to the scaling The quantization factor scale and zero point zp perform a quantization operation from full-precision float32 format to int8 format or an inverse quantization operation from int8 format to full-precision float32 format.

Example 3

This embodiment provides an electronic device, including a storage medium and a processor; the storage medium is used to store instructions; and the processor is used to operate according to the instructions to execute the method described in Embodiment 1.

Those skilled in the art will understand that embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in one process or multiple processes of the flowchart and/or one block or multiple blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

The above are only preferred embodiments of the present invention. It should be noted that those of ordinary skill in the art can also make several improvements and modifications without departing from the technical principles of the present invention. These improvements and modifications It should also be regarded as the protection scope of the present invention.

Claims (10)

1. A pulsed neural network quantification method for a brain-like computing platform, comprising:

loading and storing a pulse neural network model of the pre-training full-precision float32, and marking the pulse neural network model as a pulse neural network model T1;

the process of modifying the impulse neural network model T1 to obtain the impulse neural network model T2 is as follows:

inserting pseudo quantization nodes into an input layer and an output layer of the impulse neural network model T1, quantizing input features from a full-precision float32 format to an int8 format by the input layer, and inversely quantizing output features from the int8 format to the full-precision float32 format by the output layer;

defining a linear operator of a pulse neural network model T1 to a pseudo-quantization node, dequantizing characteristic parameters input to the pseudo-quantization node from an int8 format to a full-precision float32 format, and then quantizing the characteristic parameters back to the int8 format after calculation of the pseudo-quantization node;

changing the impulse neural network model T1 into an Euler value solution form, then folding a convolution operator, batch normalization processing and LIF neuron operators as operators, and inserting weights and activation functions of all layers in the impulse neural network model T1 into observation nodes, wherein the observation nodes are used for calculating and storing the maximum value, the minimum value and the quantization range of each group of weights under each channel to obtain an impulse neural network model T2;

training the pulse neural network model T2 in a quantization process by selecting a calibration data set to select a proper scaling quantization scale factor scale and a zero zp so as to minimize precision loss before and after quantization; and deploying the trained impulse neural network model T2 to a brain-like computing platform.

2. The method for quantification of impulse neural network according to claim 1, wherein the expression formula for changing the impulse neural network model T1 into the euler numerical solution form is:

wherein V (t) is the membrane potential at time t, V _reset Is heavySetting potential, V _threshold For the threshold potential, X (t) is the input, S (t) is the pulse at time t, Φ is the step function, τ is the time decay constant.

3. The pulsed neural network quantization method of claim 1, wherein the method of operator folding convolution operators, batch normalization processing, LIF neuron operators comprises:

the convolution operator, batch normalization processing and LIF neuron operator expression formulas of the impulse neural network model T1 are as follows:

the expression formula for folding the convolution operator, batch normalization processing and LIF neuron operator as the operator is as follows:

in the formula, w ⁿ Represents the weight of the nth layer, w ⁿ⁺¹ Represents the weight of the n+1 layer, b ⁿ⁺¹ N+1 layer bias, x ⁿ⁺² (t) represents the output value at the time of the (n+2) th layer, x ⁿ⁺¹ (t) represents the output value at the t-th time of the n+1th layer, x ^n-1 (t) represents the output value at the t-th time of the n-1 th layer, x ⁿ (t) represents the output value, σ, of the nth layer at the time t ² Variance, μmean, e=1e-5; τ is the time decay constant.

4. The method for quantizing a pulsed neural network according to claim 1, wherein selecting the calibration data set to train the quantization process on the pulsed neural network model T2 to select the appropriate scaling quantization scale and zero zp to minimize the loss of accuracy before and after quantization comprises:

training the impulse neural network model T2 by using a calibration data set, setting a step function phi in the back propagation process of the impulse neural network model T2 as a sigmoid function, calculating and storing the maximum value, the minimum value and the quantization range of each group of weights under each channel through observation nodes, and further calculating a scaling quantization scale factor and a zero zp; quantization operations to convert from full precision float32 format to int8 format or inverse quantization operations to convert from int8 format to full precision float32 format are performed according to the scaled quantization scale and zero zp.

5. The method for quantizing a pulsed neural network according to claim 4, wherein the quantization operation for converting from the full-precision float32 format to the int8 format is performed by the expression:

in the formula, x _Q For quantized output features, x is denoted as quantized input feature, and the quantized input feature x is passed through by a clip ()Quantized truncated 0 to N _levels -1 range; n (N) _levels Represented as the maximum value of the int8 format.

6. The method of claim 4, wherein the inverse quantization operation from the int8 format to the full precision float32 format is performed by the expression:

x _float ＝(x _Q -zp)scale

in the formula, x _float Representation ofOutput features for inverse quantization; x is x _Q To quantify the output characteristics.

7. The method of claim 1, wherein deploying the trained impulse neural network model T2 to the brain-like computing platform comprises: the method comprises the steps of creating a position according to one-to-one of a trained impulse neural network model T2, deploying the trained impulse neural network model T2 on a brain-like calculation platform, connecting neurons according to a mode that a neuron number is designated one by an operator after operator folding, and assigning a folded weight w to the connection ^n+1′ The method comprises the steps of carrying out a first treatment on the surface of the Setting the bias current I_offset of the neuron to b ^n+1′ And the neuron membrane capacitance C is set to τ.

8. A pulsed neural network quantification system for a brain-like computing platform, comprising:

the loading module is used for loading and storing a pulse neural network model of the pre-training full-precision float32 and is recorded as a pulse neural network model T1;

the model modification module is used for inserting pseudo quantization nodes into an input layer and an output layer of the impulse neural network model T1, quantizing input features from a full-precision float32 format to an int8 format by the input layer, and inversely quantizing output features from the int8 format to the full-precision float32 format by the output layer; defining a linear operator of a pulse neural network model T1 to a pseudo-quantization node, dequantizing characteristic parameters input to the pseudo-quantization node from an int8 format to a full-precision float32 format, and then quantizing the characteristic parameters back to the int8 format after calculation of the pseudo-quantization node; changing the impulse neural network model T1 into an Euler value solution form, then folding a convolution operator, batch normalization processing and LIF neuron operators as operators, and inserting weights and activation functions of all layers in the impulse neural network model T1 into observation nodes, wherein the observation nodes are used for calculating and storing the maximum value, the minimum value and the quantization range of each group of weights under each channel to obtain an impulse neural network model T2;

the training module is used for selecting a calibration data set to train the quantization process of the impulse neural network model T2 so as to select a proper scaling quantization scale factor scale and a zero zp to minimize the precision loss before and after quantization;

and the execution module is used for deploying the trained impulse neural network model T2 to the brain-like computing platform.

9. The impulse neural network quantization system according to claim 1, wherein the training module is configured to select the calibration data set to perform training of the quantization process on the impulse neural network model T2, so as to select the appropriate scaling quantization scale factor scale and zero zp to minimize the precision loss before and after quantization, and the method comprises:

training the impulse neural network model T2 by using a calibration data set, calculating and storing the maximum value, the minimum value and the quantization range of each group of weights under each channel through observation nodes, and further calculating a scaling quantization scale factor and a zero zp; quantization operations to convert from full precision float32 format to int8 format or inverse quantization operations to convert from int8 format to full precision float32 format are performed according to the scaled quantization scale and zero zp.

10. An electronic device comprising a storage medium and a processor; the storage medium is used for storing instructions; the processor is operative to perform the method of any one of claims 1 to 7 in accordance with the instructions.