WO2019196223A1 - Acceleration method and accelerator used for convolutional neural network - Google Patents

Abstract

The present invention provides an acceleration method and accelerator used for a convolutional neural network, the method comprising: S1, for any layer of the convolutional neural network, calculating the density of various feature maps output by this layer; S2, comparing the density of the various feature maps output by this layer to a plurality of preset thresholds, and performing, according to the comparison results, sparse coding to the various feature maps, different comparison results corresponding to different sparse coding schemes; and S3, performing, on the basis of the convolution layer of a layer following this layer, convolution to the various sparse coded feature maps and various pre-sparse coded convolution kernels in the convolutional neural network. The present invention reduces the calculated amount of the convolution operation in the convolutional neural network and improves operation speed.

Description

Acceleration method and accelerator applied to convolutional neural network

cross reference

The present application is hereby incorporated by reference in its entirety in its entirety in its entirety in its entirety in the the the the the the the the the the the the the

Technical field

The invention belongs to the technical field of operation optimization, and more particularly to an acceleration method and an accelerator applied to a convolutional neural network.

Background technique

The Convolutional Neural Network (CNN) is a feedforward neural network whose artificial neurons can respond to surrounding units in a part of the coverage and is suitable for processing large images. Convolutional neural networks are widely used in image recognition, speech recognition and other fields, but the amount of calculation is very large.

Because the activation function ReLU (Rectified Linear Unit) in the convolutional neural network will cause a large number of sparse feature maps; at the same time, training the convolutional neural network by pruning and other methods will result in a large amount of sparse weight data. (weight data). The use of feature maps and the sparseness of weight data can greatly improve the computational efficiency of convolutional neural networks. At present, there are many methods to improve the calculation speed based on the sparsity of feature maps and weight data in convolutional neural networks. These methods can be roughly divided into two categories, one focusing on skipping the 0 value. For example, some methods remove the 0 value in the input, thereby reducing the invalid calculation of input 0. The other type takes a method of ignoring zero values. For example, some methods do not perform a multiplication operation when the input data is 0, thereby reducing the operation. However, these methods are all focused on dealing with the sparse neural network itself, assuming that the neural network is sparse. However, in fact, the output feature maps of each layer in the convolutional neural network may be sparse and may be non-sparse. In practice, the weight data of each layer of the convolutional neural network and the density of the feature map are generally distributed between 5% and 90%.

A sparse matrix is a matrix in which the number of elements with an index value of 0 is much larger than the number of non-zero elements, and the non-zero element distribution has no regular matrix. In the prior art, only a sparse convolutional neural network can be processed on the one hand, and the calculation amount is large and the operation speed is low in the case where the convolutional neural network is not sparse; on the other hand, the prior art can only process the convolutional nerve. The weight data or feature map is sparse and cannot handle the case where the weight data and the feature map are sparse.

Summary of the invention

In order to overcome the above problem of low computation speed of the convolutional neural network or at least partially solve the above problems, the present invention provides an acceleration method and an accelerator applied to a convolutional neural network.

According to a first aspect of the present invention, there is provided an acceleration method applied to a convolutional neural network, comprising:

S1, for any layer in the convolutional neural network, respectively calculating the density of each feature map outputted by the layer;

S2, comparing the density of each of the feature maps outputted by the layer with a plurality of preset thresholds, and performing sparse coding on each of the feature maps according to the comparison result; wherein different comparison results correspond to different sparse coding modes;

S3, convolving each of the feature maps after sparse coding and each convolution kernel in the pre-sparse coded convolutional neural network based on a convolution layer of a layer below the layer.

Specifically, the step S1 specifically includes:

For any of the feature maps, the number of non-zero elements in the feature map and the total number of all elements in the feature map are counted;

The ratio between the number of non-zero elements in the feature map and the total number of all elements in the feature map is taken as the density of the feature map.

Specifically, the preset threshold includes a first preset threshold and a second preset threshold; wherein the first preset threshold is smaller than the second preset threshold;

Correspondingly, the step S2 specifically includes:

If the density of each of the feature maps is less than the first preset threshold, encoding each of the feature maps into a sparse matrix storage format;

If the density of each of the feature maps is greater than or equal to the first preset threshold and less than the second preset threshold, marking 0 elements in each of the feature maps;

If the density of each of the feature maps is greater than or equal to the second predetermined threshold, each of the feature maps is not sparsely encoded.

Specifically, before the step S3, the method further includes:

Calculating the density of each convolution kernel in the trained convolutional network;

If each of the convolution kernels has a density less than the first predetermined threshold, encoding each of the convolution kernels into a sparse matrix storage format;

If the density of each of the convolution kernels is greater than or equal to the first predetermined threshold and less than the second predetermined threshold, marking 0 elements in each of the convolution kernels;

If the density of each of the convolution kernels is greater than or equal to the second predetermined threshold, each of the convolution kernels is not sparsely encoded.

Specifically, the step S3 specifically includes:

When the mark exists in each of the feature maps or each of the convolution kernels, no calculation is performed on each of the feature maps or the elements corresponding to the marks in each of the convolution kernels.

According to another aspect of the present invention, an accelerator for a convolutional neural network is provided, comprising: a neural network computing array module and a dynamic sparse adjustment module;

The dynamic sparse adjustment module is configured to calculate the density of each feature image outputted by each layer of the convolutional neural network, compare the density of each of the feature maps with a plurality of preset thresholds, and compare the results according to the comparison results. The feature map is subjected to sparse coding; wherein different comparison results correspond to different sparse coding modes;

The neural network computing array module is configured to perform a convolution operation on each of the sparsely encoded feature maps and the pre-sparsely encoded convolutional neural networks.

Specifically, the dynamic sparse adjustment module includes an online density identification module, an output temporary registration module, a dynamic coding module, and a dynamic sparse control module;

The online density identification module is configured to count, for any of the feature maps, the number of 0 elements in the feature map and the total number of all elements in the feature map; The ratio between the number and the total number of all elements in the feature map as the density of the feature map;

The output temporary registration module is configured to store each of the feature maps output by each layer in the convolutional neural network;

The dynamic sparse control module is configured to compare the density of each of the feature maps output by the online density identification module with a plurality of preset thresholds;

The dynamic encoding module is configured to perform sparse encoding on each of the feature maps in the output temporary registration module according to a comparison result.

Specifically, the preset threshold includes a first preset threshold and a second preset threshold; wherein the first preset threshold is smaller than the second preset threshold;

Correspondingly, the dynamic coding module is specifically configured to:

If the density of each of the feature maps is less than the first preset threshold, encoding each of the feature maps into a sparse matrix storage format;

If the density of each of the feature maps is greater than or equal to the first preset threshold and less than the second preset threshold, marking 0 elements in each of the feature maps;

If the density of each of the feature maps is greater than or equal to the second predetermined threshold, each of the feature maps is not sparsely encoded.

Specifically, the dynamic coding module is further configured to:

If each of the pre-computed convolution kernels has a density less than the first predetermined threshold, encoding each of the convolution kernels into a sparse matrix storage format;

If the density of each of the convolution kernels is greater than or equal to the first predetermined threshold and less than the second predetermined threshold, marking 0 elements in each of the convolution kernels;

If the density of each of the convolution kernels is greater than or equal to the second predetermined threshold, each of the convolution kernels is not sparsely encoded.

Specifically, the neural network computing array module is specifically configured to:

When the mark exists in each of the feature maps or each of the convolution kernels, no calculation is performed on each of the feature maps or the elements corresponding to the marks in each of the convolution kernels.

The present invention provides an acceleration method and an accelerator applied to a convolutional neural network, which obtains each of the features by comparing the density of each feature image outputted by each layer in the convolutional neural network with a plurality of preset thresholds. The sparse state of the graph, the feature maps of different sparse states are sparsely coded in different ways, and then based on the thinned coded feature maps of the next layer of each layer and the volumes in the pre-sparse coded convolutional neural network The product core performs a convolution operation to reduce the amount of calculation of the convolution operation in the convolutional neural network and to increase the operation speed.

DRAWINGS

1 is a schematic overall flow chart of an acceleration method applied to a convolutional neural network according to an embodiment of the present invention;

2 is a schematic diagram of an overall structure of an accelerator applied to a convolutional neural network according to an embodiment of the present invention;

3 is a schematic diagram of a limit energy efficiency test result of an accelerator applied to a convolutional neural network according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of comparison of extreme energy efficiency test results in an accelerator applied to a convolutional neural network according to an embodiment of the present invention.

detailed description

The specific embodiments of the present invention are further described in detail below with reference to the drawings and embodiments. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.

An acceleration method for a convolutional neural network is provided in an embodiment of the present invention. FIG. 1 is a schematic flowchart of an overall acceleration method applied to a convolutional neural network according to an embodiment of the present invention, where the method includes:

S1, for any layer in the convolutional neural network, respectively calculating the density of each feature map outputted by the layer;

Specifically, the convolutional neural network may include or not include a pooling layer. First, the convolutional neural network is trained. After the training is completed, the convolution kernel in the convolutional neural network no longer changes. Therefore, the convolution kernel in the convolutional neural network does not need online dynamic sparse coding, and direct offline sparse coding. Just one time. The line refers to the chip on the accelerator, and the line below refers to the chip not located on the accelerator. The convolutional calculation is performed by directly reading the sparsely encoded convolution kernel at each convolution operation. When the original image data is input, the original image data is sparsely encoded, and then the sparsely encoded raw data and the sparsely encoded convolution kernel are input to the first layer convolutional layer of the convolutional neural network for convolution calculation. Since the original image data is generally not sparse, the original image data may not be sparsely encoded, and the original image data may be directly input. The sparse coding is to store the data in a sparse format.

In S1, since the density of each characteristic map outputted by each layer in the convolutional neural network is different, the characteristic maps of the output of different layers are also dynamically changed, so the density is also dynamically changed. The density indicates the degree of sparsity of each of the feature maps. In order to better improve the operation speed of the convolutional neural network, the density of each feature image outputted by each layer is calculated, and each characteristic image outputted by each layer is obtained according to the density of each feature image outputted by each layer. Perform sparse coding.

S2, comparing the density of each of the feature maps outputted by the layer with a plurality of preset thresholds, and performing sparse coding on each of the feature maps according to the comparison result; wherein different comparison results correspond to different sparse coding modes;

In S2, in the prior art, all the feature maps output by each layer are sparsely encoded, and the calculation amount is large. In this embodiment, the sparse state of each of the feature maps output by the layer is obtained according to the preset threshold. Therefore, the feature maps of different sparse states are subjected to different forms of sparse coding.

S3, convolving each of the feature maps after sparse coding and each convolution kernel in the pre-sparse coded convolutional neural network based on a convolution layer of a layer below the layer.

In S3, each of the feature maps after sparse coding and each convolution kernel in the pre-sparse coded convolutional neural network are used as input of a convolution layer of a layer below the layer, and a convolution operation is performed. Then, the result of the convolution operation is used as an input to the next layer of the convolution layer, and the feature map outputted by the next layer of the convolution layer is continued to perform the above-described sparse coding and convolution operations until the convolution The last layer in the neural network outputs the feature maps. This embodiment is not limited to the sparse coding mode of the convolution kernel.

In this embodiment, the density of each feature image outputted by each layer in the convolutional neural network is compared with a plurality of preset thresholds, and the sparse state of each of the feature maps is obtained, and the feature maps of different sparse states are sparse in different manners. Coding, and then convolving the sparse-coded feature maps and the convolution kernels in the pre-sparse-coded convolutional neural network based on the convolutional layer of the next layer of each layer to reduce the convolution operation in the convolutional neural network. Calculate the amount and increase the speed of the operation.

On the basis of the foregoing embodiment, in the embodiment, the step S1 specifically includes: counting the number of non-zero elements in the feature map and the total number of all the elements in the feature map for any of the feature maps. The ratio between the number of non-zero elements in the feature map and the total number of all elements in the feature map is taken as the density of the feature map.

Specifically, the density of each feature map is a ratio between the number of non-zero elements in each feature map and the total number of all elements in each feature map. For example, if the number of non-zero elements in a feature map is 10, and the total number of all elements in the feature map is 100, the density of the feature map is 0.1.

On the basis of the foregoing embodiment, the preset threshold includes a first preset threshold and a second preset threshold; the preset threshold includes a first preset threshold and a second preset threshold; The step S2 specifically includes: if the density of each of the feature maps is less than the first preset threshold, encoding each of the feature maps into a sparse matrix storage format; if each of the feature maps has a density greater than Or equal to the first preset threshold, and less than the second preset threshold, marking 0 elements in each of the feature maps; if the density of each of the feature maps is greater than or equal to the second The preset threshold is not sparsely encoded for each of the feature maps.

Specifically, the preset threshold in this embodiment includes a first preset threshold th1 and a second preset threshold th2. Dividing the feature state AS of each of the feature maps into three states according to the first preset threshold and the second preset threshold, that is, the feature map whose density is smaller than the first preset threshold is completely sparse a state S, wherein the feature map having a density greater than or equal to the first preset threshold and less than the second preset threshold is divided into a medium sparse state M, and the density is greater than or equal to the second preset threshold. The feature map is divided into a completely non-sparse state D. If each feature map is a sparse state S, each of the feature maps is encoded into a sparse matrix storage format, where the sparse matrix storage format includes non-zero data activ and sparse index index in each of the feature maps, such as coordinate encoding and Compress sparse row coding. By encoding the feature map as a sparse matrix storage format, a large amount of storage space can be saved while saving a lot of computation time. If each feature map is a medium sparse state M, the 0 element in each feature map is marked with a guard, which is used to identify the 0 element. Elements that are tagged may not participate in calculations and storage, thereby reducing power consumption. Marking the 0 elements in each feature map is also a sparse coding method. If each feature map is completely non-sparse state D, dynamic coding is not required, and non-sparse data of each feature map is directly output.

On the basis of the foregoing embodiment, before step S3 in the embodiment, the method further includes: calculating a density of each convolution kernel in the trained convolution network; if each of the convolution kernels has a density less than the first a preset threshold, the each convolution kernel is encoded into a sparse matrix storage format; if the density of each of the convolution kernels is greater than or equal to the first preset threshold, and less than the second preset threshold And marking 0 elements in each of the convolution kernels; if each of the convolution kernels has a density greater than or equal to the second predetermined threshold, each of the convolution kernels is not encoded.

Specifically, the density of each convolution kernel is the ratio between the number of non-zero elements in each convolution kernel and the total number of all elements in each convolution kernel. The state WS of each convolution kernel is divided into three states as in the feature map. Each state corresponds to a different sparse coding scheme. Since there are three states in the feature map and the convolution kernel, there are 9 states after the combination, so that the density of the convolutional neural network is more finely divided.

On the basis of the above embodiments, the step S3 in the embodiment specifically includes: when each of the feature maps or each of the convolution kernels has the mark, each of the feature maps or each The elements corresponding to the tags in the convolution kernel are not evaluated.

Specifically, when each of the feature maps or each of the convolution kernels is in a completely sparse state S, remove 0 before input, reduce storage space, and do not calculate 0 elements; when each of the feature maps or each When the convolution kernel is in the medium sparse state M, although the 0 elements in each of the feature maps or the respective convolution kernels are stored, the elements corresponding to the markers are not calculated, thereby reducing the calculation.

In another embodiment of the present invention, an accelerator applied to a convolutional neural network is provided. FIG. 2 is a schematic diagram of an overall structure of an accelerator applied to a convolutional neural network according to an embodiment of the present invention, including: a neural network computing array module and a dynamic sparse adjustment module, wherein the dynamic sparse adjustment module is configured to calculate a density of each feature image outputted by each layer in the convolutional neural network, and compare the density of each of the feature maps with a plurality of preset thresholds, Performing sparse coding on each of the feature maps according to the comparison result; wherein different comparison results correspond to different sparse coding modes; the neural network calculation array module is configured to perform sparsely coded each of the feature maps and pre-sparse coding Each convolution kernel in the convolutional neural network performs a convolution operation.

Specifically, the convolutional neural network may include or not include a pooling layer. Firstly, the convolutional neural network is trained. After the training is completed, the convolution kernel in the convolutional neural network no longer changes. Therefore, the convolution kernel in the convolutional neural network does not need online dynamic sparse coding, and the direct offline sparse coding once can. The neural network computational array module directly reads the offline sparsely encoded convolution kernel for convolution calculations each time a convolution operation is performed. When the convolutional neural network inputs the original image data, the dynamic sparse adjustment module sparsely encodes the original image data, and then the neural network calculation array module performs convolution calculation according to the sparsely encoded original data and the sparsely encoded convolution kernel. Since the original image data is generally not sparse, the original image data may not be sparsely encoded, and the original image data may be directly input. The sparse coding is to store the data in a sparse format.

Since the density of each characteristic map outputted by each layer in the convolutional neural network is different, the characteristic maps of the output of different layers are also dynamically changed, so the density is also dynamically changed. The density indicates the degree of sparsity of each of the feature maps. In order to better improve the operation speed of the convolutional neural network, the dynamic sparse adjustment module calculates the density of each feature map outputted by each layer, according to the density of each feature map output by each layer. Each feature map output by the layer is sparsely encoded.

The dynamic sparse adjustment module acquires a sparse state of each of the feature maps output by the layer according to a plurality of preset thresholds. Therefore, the feature maps of different sparse states are subjected to different forms of sparse coding, and are not limited to one type of sparse coding. In the prior art, all the special maps output by each layer are sparsely encoded, and the calculation amount is large.

The neural network computing array module performs a convolution operation according to each of the feature maps after sparse coding and each convolution kernel in the pre-sparse coded convolutional neural network. If the pooling module is included, the pooling module performs a pooling operation on the result of the convolution operation. In addition, the accelerator further includes an intermediate data memory module, a main chip controller, and an on-chip data exchange module. Wherein, the main controller controls the running action and timing of the entire chip of the accelerator. The on-chip data exchange module is used to read data from the chip external memory or write the chip-calculated data to external storage. For example, after initialization, the chip reads the original image data and the initial convolution kernel from the external memory through the chip upper and lower data exchange modules under the control of the main controller. The intermediate data storage module is configured to store intermediate results in the calculation process of the neural network computing array module.

In this embodiment, the dynamic sparse adjustment module compares the density of each feature image outputted by each layer of the convolutional neural network with a plurality of preset thresholds, obtains a sparse state of each of the feature maps, and differentizes the feature maps of different sparse states. Sparse coding of the method for the neural network computing array module to perform convolution operations on the sparsely encoded feature maps and the convolution kernels in the pre-sparse coded convolutional neural network, on the one hand, reducing convolution in convolutional neural networks The calculation amount of the operation increases the calculation speed; on the other hand, according to the sparse state, the processing state of the accelerator is dynamically switched, and the flexibility of the accelerator is improved.

On the basis of the foregoing embodiment, the dynamic sparse adjustment module in the embodiment includes an online density identification module, an output temporary registration module, a dynamic coding module, and a dynamic sparse control module; wherein the online density identification module For counting any of the feature maps, counting the number of 0 elements in the feature map and the total number of all the elements in the feature map; the number of 0 elements in the feature map and all the elements in the feature map The ratio between the total number is used as the density of the feature map; the output temporary registration module is configured to store each of the feature maps output by each layer in the convolutional neural network; the dynamic sparse control module is configured to The density of each of the feature maps output by the online density identification module is compared with a plurality of preset thresholds; the dynamic encoding module is configured to sparse each of the feature maps in the output temporary registration module according to a comparison result. coding.

Specifically, the dynamic sparse adjustment module specifically includes four modules. The online density identification module is used to count the number of non-zero elements in each feature map during the calculation process to calculate the density of each feature map. The output temporary registration module is configured to temporarily store the feature map outputted by each layer in the convolutional neural network in a non-sparse format. The dynamic sparse control module is configured to control a sparse state of the feature map by using a preset plurality of preset thresholds. The dynamic encoding module performs sparse encoding on each of the feature maps in the output temporary registration module according to a sparse state of each of the feature maps, thereby increasing the speed of the convolution operation.

On the basis of the foregoing embodiment, the preset threshold includes a first preset threshold and a second preset threshold; the preset threshold includes a first preset threshold and a second preset threshold; The dynamic coding module is specifically configured to: if each of the feature maps has a density less than the first preset threshold, encode each of the feature maps into a sparse matrix storage format; if each of the feature maps is dense If the degree is greater than or equal to the first preset threshold, and is less than the second preset threshold, the 0 elements in each of the feature maps are marked; if the density of each of the feature maps is greater than or equal to the The second preset threshold does not encode each of the feature maps.

Specifically, the preset threshold in this embodiment includes a first preset threshold th1 and a second preset threshold th2. The dynamic sparse control module divides the feature state AS of each of the feature maps into three states according to the first preset threshold and the second preset threshold, that is, the density is less than the first preset threshold. The feature map is divided into a completely sparse state S, a feature map having a density greater than or equal to the first predetermined threshold, and a feature map smaller than the second predetermined threshold is classified into a medium sparse state M, and the density is greater than or equal to the The feature map of the second preset threshold is divided into a completely non-sparse state D.

If each feature map is a sparse state S, the dynamic coding module encodes each of the feature maps in the output temporary registration module into a sparse matrix storage format, where the sparse matrix storage format includes non-characteristics in each of the feature maps. 0 data activ and sparse index index, such as coordinate encoding and compressed sparse row encoding. By encoding the feature map as a sparse matrix storage format, a large amount of storage space can be saved while saving a lot of computation time. If each feature map is a medium sparse state M, the dynamic coding module adds a flag guard to the 0 element in each feature map in the output temporary registration module, and may not participate in calculation and storage for the marked element, thereby reducing power consumption. . If each feature map is completely non-sparse state D, dynamic coding is not required, and the dynamic coding module directly outputs non-sparse data of each feature map.

On the basis of the foregoing embodiment, the dynamic coding module in this embodiment is further configured to: if the density of each of the convolution kernels calculated in advance is less than the first preset threshold, each of the convolutions The kernel is encoded as a sparse matrix storage format; if the density of each of the convolution kernels is greater than or equal to the first predetermined threshold and less than the second predetermined threshold, then 0 of each of the convolution kernels The element is marked; if the density of each of the convolution kernels is greater than or equal to the second predetermined threshold, each of the convolution kernels is not encoded.

Specifically, the density of each convolution kernel is the ratio between the number of non-zero elements in each convolution kernel and the total number of all elements in each convolution kernel. The state WS of each convolution kernel has three states as in the feature map. Each state corresponds to a different sparse coding scheme. Since there are three states in the feature map and the convolution kernel, there are 9 states after the combination, so that the density of the convolutional neural network is more finely divided.

On the basis of the foregoing embodiments, the neural network calculation array module in the embodiment is specifically configured to: when each of the feature maps or each of the convolution kernels has the label, Or the corresponding element in each of the convolution kernels is not calculated.

Specifically, when each of the feature maps or each of the convolution kernels is in a completely sparse state S, the 0 is reduced before each of the feature maps or each of the convolution kernels is input into the neural network calculation array module. The storage space is not calculated at the same time; when each of the feature maps or each of the convolution kernels is in a medium sparse state M, the 0 elements in each of the feature maps or each of the convolution kernels are stored, However, the elements corresponding to the mark are not calculated, thereby reducing the calculation.

For example, the chip of the accelerator is fabricated by a TSMC 65 nm process, the chip having an area of 3 mm*4 mm, an operating frequency of 20-200 MHz, and a power consumption of 20.5-248.4 mW. In this embodiment, the ultimate energy efficiency rises rapidly as the feature map and the convolution kernel density decrease, as shown in FIG. When the density of the feature map and the convolution kernel are both 5%, the ultimate energy efficiency can reach 62.1 TOPS/W, which is 6.2 times the ultimate energy efficiency when the accelerator of the embodiment is not used. As shown in FIG. 4, the energy efficiency of the embodiment can be increased by 4.3 times compared to the implementation of only supporting feature data sparse. Compared to the implementation without adaptive sparse control, the energy efficiency of the present invention can be increased by 2.8 times. The energy efficiency of the present invention can be increased by a factor of 2 compared to the implementation without variable density control but variable quantization accuracy.

Finally, the method of the present application is only a preferred embodiment and is not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims (10)

An acceleration method applied to a convolutional neural network, comprising: S1, for any layer in the convolutional neural network, respectively calculating the density of each feature map outputted by the layer; S2, comparing the density of each of the feature maps outputted by the layer with a plurality of preset thresholds, and performing sparse coding on each of the feature maps according to the comparison result; wherein different comparison results correspond to different sparse coding modes; S3, convolving each of the feature maps after sparse coding and each convolution kernel in the pre-sparse coded convolutional neural network based on a convolution layer of a layer below the layer. The method according to claim 1, wherein the step S1 specifically comprises: For any of the feature maps, the number of non-zero elements in the feature map and the total number of all elements in the feature map are counted; The ratio between the number of non-zero elements in the feature map and the total number of all elements in the feature map is taken as the density of the feature map. The method according to claim 1, wherein the preset threshold includes a first preset threshold and a second preset threshold; wherein the first preset threshold is smaller than the second preset threshold; Correspondingly, the step S2 specifically includes: If the density of each of the feature maps is less than the first preset threshold, encoding each of the feature maps into a sparse matrix storage format; If the density of each of the feature maps is greater than or equal to the first preset threshold and less than the second preset threshold, marking 0 elements in each of the feature maps; If the density of each of the feature maps is greater than or equal to the second predetermined threshold, each of the feature maps is not sparsely encoded. The method according to claim 3, wherein the step S3 further comprises: Calculating the density of each convolution kernel in the trained convolutional network; If each of the convolution kernels has a density less than the first predetermined threshold, encoding each of the convolution kernels into a sparse matrix storage format; If the density of each of the convolution kernels is greater than or equal to the first predetermined threshold and less than the second predetermined threshold, marking 0 elements in each of the convolution kernels; If the density of each of the convolution kernels is greater than or equal to the second predetermined threshold, each of the convolution kernels is not sparsely encoded. The method according to claim 3 or 4, wherein the step S3 specifically comprises: When the mark exists in each of the feature maps or each of the convolution kernels, no calculation is performed on each of the feature maps or the elements corresponding to the marks in each of the convolution kernels. An accelerator applied to a convolutional neural network, comprising: a neural network computing array module and a dynamic sparse adjustment module; The dynamic sparse adjustment module is configured to calculate the density of each feature image outputted by each layer in the convolutional neural network, compare the density of each of the feature maps with a plurality of preset thresholds, and compare the results according to the comparison results. The feature map is subjected to sparse coding; wherein different comparison results correspond to different sparse coding modes; The neural network computing array module is configured to perform a convolution operation on each of the sparsely encoded feature maps and the pre-sparsely encoded convolutional neural networks. The accelerator according to claim 6, wherein the dynamic sparse adjustment module comprises an online density identification module, an output temporary registration module, a dynamic coding module, and a dynamic sparse control module; The online density identification module is configured to count, for any of the feature maps, the number of 0 elements in the feature map and the total number of all elements in the feature map; The ratio between the number and the total number of all elements in the feature map as the density of the feature map; The output temporary registration module is configured to store each of the feature maps output by each layer in the convolutional neural network; The dynamic sparse control module is configured to compare the density of each of the feature maps output by the online density identification module with a plurality of preset thresholds; The dynamic encoding module is configured to perform sparse encoding on each of the feature maps in the output temporary registration module according to a comparison result. The accelerator according to claim 7, wherein the preset threshold comprises a first preset threshold and a second preset threshold; wherein the first preset threshold is smaller than the second preset threshold; Correspondingly, the dynamic coding module is specifically configured to: If the density of each of the feature maps is less than the first preset threshold, encoding each of the feature maps into a sparse matrix storage format; If the density of each of the feature maps is greater than or equal to the first preset threshold and less than the second preset threshold, marking 0 elements in each of the feature maps; If the density of each of the feature maps is greater than or equal to the second predetermined threshold, each of the feature maps is not sparsely encoded. The accelerator according to claim 8, wherein the dynamic encoding module is further configured to: If each of the pre-computed convolution kernels has a density less than the first predetermined threshold, encoding each of the convolution kernels into a sparse matrix storage format; If the density of each of the convolution kernels is greater than or equal to the first predetermined threshold and less than the second predetermined threshold, marking 0 elements in each of the convolution kernels; If the density of each of the convolution kernels is greater than or equal to the second predetermined threshold, each of the convolution kernels is not sparsely encoded. The accelerator according to claim 7 or 8, wherein the neural network computing array module is specifically configured to: When the mark exists in each of the feature maps or each of the convolution kernels, no calculation is performed on each of the feature maps or the elements corresponding to the marks in each of the convolution kernels.

Images