CN111612137A - Optimization method and system of convolutional neural network based on soft threshold ternary parameters - Google Patents

Abstract

The invention belongs to the field of data processing, and particularly relates to a soft threshold ternary parameter-based convolutional neural network optimization method and system, aiming at solving the problem of optimization acceleration of a convolutional neural network. The method comprises the following steps: splitting the convolutional layer of the convolutional neural network into two parallel convolutional layers with the same size; and after the two convolution layers are respectively binarized under the constraint condition with equal scale coefficients, the two convolution layers are correspondingly added to obtain a ternary parameter. The invention can realize the optimized acceleration and compression of the deep convolutional neural network.

Description

Optimization method and system of convolutional neural network based on soft threshold ternary parameters

technical field

The invention belongs to the field of data processing, and in particular relates to a convolutional neural network optimization method and system based on soft threshold ternary parameters.

Background technique

In recent years, deep convolutional neural networks have made great breakthroughs in many fields such as computer vision, speech processing, and machine learning, significantly improving the performance of machine algorithms in multiple tasks such as image classification, object detection, and speech recognition. And in the Internet, video surveillance and other industries have been widely used.

The training process of deep convolutional neural network is to learn and adjust network parameters based on large-scale datasets containing manual annotation information. In general, large-capacity, high-complexity deep convolutional networks can learn more comprehensively from the data, resulting in better performance metrics. However, with the increase of the number of network layers and parameters, the computational and storage costs will increase significantly. Therefore, at present, most of the training and testing of convolutional neural networks can only be performed on high-performance computing clusters.

On the other hand, mobile Internet technology has made great progress in recent years, and its application in real life has become more and more extensive. In the application scenario of the mobile Internet, the devices used by users, such as mobile phones or tablet computers, have very limited computing and storage capabilities. Although deep convolutional neural networks can be trained on computing clusters, in mobile platform application scenarios, the testing process of network models still needs to be performed on mobile devices, which poses two challenges: how to reduce the performance of convolutional neural networks. Inference time, and how to compress the storage overhead of the network model.

For the acceleration and compression of convolutional neural networks, some effective ternary algorithms have been proposed. These algorithms set two fixed thresholds by solving optimization problems or gradient learning, and achieve the purpose of ternarization through fixed thresholds. Such methods can be uniformly classified as "hard threshold" ternarization methods. However, these hard-threshold ternarization algorithms often take a lot of time to calculate the optimal threshold due to the difficulty of threshold calculation. Therefore, how to efficiently perform ternaryization of deep convolutional neural networks remains to be studied.

SUMMARY OF THE INVENTION

In order to solve the above problems in the prior art, that is, in order to solve the optimization acceleration problem of convolutional neural networks, the first aspect of the present invention provides a convolutional neural network optimization method based on soft threshold ternary parameters, including constructing The intermediate network and network optimization are two parts:

Building an intermediate network includes the following steps:

Step S100, each convolution operation layer of the original convolutional neural network is divided into two parallel sub-layers with the same convolution kernel size, as the alienated convolution operation layer; the convolution operation layer includes convolution layers and fully connected layers;

Step S200, under the constraint of the same equal-scale coefficients, binarize the sub-layers in each alienated convolution operation layer to obtain an intermediate network;

Network optimization, including the following steps:

The intermediate network is trained based on the training data to obtain an optimized intermediate network;

The binarization weights of the two sublayers in each alienated convolution operation layer in the optimized intermediate network are added to obtain a ternary weight, and a ternary convolutional neural network is obtained.

In some preferred embodiments, the two sub-layers are arranged side by side in position, and the shape is equal to the size of the original convolution kernel.

In some preferred embodiments, the convolution operation layer includes layers 2 to L-1 of the L-layer convolutional neural network; wherein L is the total number of layers of the convolutional neural network.

In some preferred embodiments, the output of the alienated convolution operation layer is equal to the input of the layer and the convolution kernels of the two parallel sub-layers are respectively added after convolution operation.

In some preferred embodiments, in step S200, "binarize the sublayers in each alienated convolution operation layer", the method is as follows:

It is solved by the optimization problem of minimizing the quantization error between the full-precision weights W ₁ , W ₂ and the binarization weights B ₁ , B ₂ ; where W ₁ and W ₂ are the two sub-layers in the alienated convolution operation layer, respectively. The full-precision weight of the layer, B ₁ and B ₂ are the binarization weights of the two sub-layers in the alienated convolution operation layer, respectively.

In some preferred embodiments, the constraint condition for solving the optimization problem is that the equal-scale coefficients α ₁ and α ₂ of the binarization weights of the two sub-layers in the alienated convolution operation layer are equal.

In some preferred embodiments, in the ternarized convolutional neural network, an activation quantization layer Ternarize() is set before each convolution operation layer, and the input X _i of this layer is subjected to ternarization processing:

为三值化处理的输出。in,

is the output of the ternarization process.

In another aspect of the present invention, a convolutional neural network optimization system based on soft threshold ternary parameters is proposed, which includes constructing an intermediate network module and a network optimization module;

The building intermediate network module is configured to build an intermediate network through the following methods:

Step S100, each convolution operation layer of the original convolutional neural network is divided into two parallel sub-layers with the same convolution kernel size, as the alienated convolution operation layer; the convolution operation layer includes convolution layers and fully connected layers;

Step S200, under the constraint of the same equal-scale coefficients, binarize the sub-layers in each alienated convolution operation layer to obtain an intermediate network;

The network optimization module is configured to train the intermediate network based on the training data, obtain an optimized intermediate network, and binarize the two sublayers in each alienated convolution operation layer in the optimized intermediate network. The weights are added to obtain a three-valued weight to obtain a three-valued convolutional neural network.

In a third aspect of the present invention, a storage device is provided, in which a plurality of programs are stored, and the programs are adapted to be loaded and executed by a processor to realize the above-mentioned optimization method of a convolutional neural network based on a soft-threshold ternary parameter .

In a fourth aspect of the present invention, a processing device is provided, including a processor and a storage device; the processor is adapted to execute various programs; the storage device is adapted to store multiple programs; the programs are adapted to be loaded by the processor And execute to realize the above-mentioned optimization method of convolutional neural network based on soft threshold ternary parameter.

Beneficial effects of the present invention:

The convolutional neural network optimization method based on the soft-threshold ternary parameter of the present invention divides the convolutional kernel of the convolutional neural network into two parallel convolutional layers of the same size, and then divides the convolutional neural network into two parallel convolutional layers with equal scale coefficients. Under the constraints, binarize the two convolutional layers respectively and add them correspondingly to obtain three-valued parameters, and use the bit operation to replace the original floating-point convolution operation, so that the optimization and acceleration of the deep convolutional neural network can be achieved. compression.

Description of drawings

Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

1 is a schematic flowchart of a convolutional neural network optimization method based on a soft threshold ternary parameter according to an embodiment of the present invention;

Figure 2 is a schematic diagram of the image classification process of a deep convolutional neural network;

3 is a schematic diagram of the convolution operation of a deep convolutional neural network in an image classification process;

4 is a schematic diagram of a process of adding parallel binarized convolution kernels to obtain a ternary convolution kernel in an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not All examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the related invention, but not to limit the invention. In addition, it should be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.

It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other in the case of no conflict.

A convolutional neural network optimization method based on soft threshold ternary parameters of the present invention, as shown in Figure 1, includes two parts: constructing an intermediate network and optimizing the network:

Building an intermediate network includes the following steps:

Step S100, each convolution operation layer of the original convolutional neural network is divided into two parallel sub-layers with the same convolution kernel size, as the alienated convolution operation layer; the convolution operation layer includes convolution layers and fully connected layers;

Step S200, under the constraint of the same equal-scale coefficients, binarize the sub-layers in each alienated convolution operation layer to obtain an intermediate network;

Network optimization, including the following steps:

The intermediate network is trained based on the training data to obtain an optimized intermediate network;

The binarization weights of the two sublayers in each alienated convolution operation layer in the optimized intermediate network are added to obtain a ternary weight, and a ternary convolutional neural network is obtained.

In order to more clearly describe the convolutional neural network optimization method based on the soft threshold ternary parameter of the present invention, the following describes the steps in an embodiment of the present invention method in detail by taking a deep convolutional neural network as an example with reference to the accompanying drawings. described.

Figure 2 exemplarily shows the process of using a deep convolutional neural network for image classification. Among them, the convolutional neural network contains multiple convolutional layers and multiple fully connected layers. The input image is processed by the convolutional layer and the fully connected layer to obtain the classification result.

FIG. 3 exemplarily shows convolutional operations of convolutional layers in a deep convolutional neural network during image classification. Among them, each convolution layer has a set of convolution kernels, and the group of convolution kernels together constitute the weight tensor of the layer. For example, the convolution kernel can be set to 3 × 3; the processing method of the convolution layer is to use the The convolution kernel performs a convolution operation on the input feature map of the layer (that is, multiplies each convolution kernel with the corresponding elements of the convolution area at each position of the input feature map, and sums them up) to obtain the output features of the corresponding layer. picture.

A method for optimizing a convolutional neural network based on a soft-threshold ternary parameter according to an embodiment of the present invention includes constructing an intermediate network and optimizing the network.

1. Build an intermediate network

Step S100, each convolution operation layer of the original convolutional neural network is divided into two parallel sub-layers with the same convolution kernel size, as the alienated convolution operation layer; the convolution operation layer includes convolution layers and fully connected layers.

For a given convolutional neural network, all the layers with convolution operations (including the convolution layer and the fully connected layer, referred to as the convolution operation layer) w except the first layer and the last layer are split into two Parallel convolutional layers (sublayers) w1 and w2 with exactly the same shape as the original. That is, if the total number of layers of the convolutional neural network is L, the convolutional operation layer includes layers 2 to L-1 of the L-layered convolutional neural network.

In this embodiment, two sub-layers are arranged side by side in position, and the shape is equal to the size of the original convolution kernel; the output of the obtained alienated convolution operation layer is equal to the input of this layer and the convolution kernel of the two parallel sub-layers respectively. After the convolution operation is performed, they are added.

Step S200 , under the constraints of the same equal-scale coefficients, binarize the sub-layers in each alienated convolution operation layer to obtain an intermediate network.

In this step, the ternary weight T is used to approximate the full-precision weight, that is, each convolutional layer in the original network is composed of the ternary weight T ∈ {-1, 0, +1} obtained by solving the optimization problem to fit.

(1) Binarize the full-precision weights W ₁ and W ₂ of the two sub-layers obtained in step S100 respectively, that is, use the binary weights B ₁ , B ₂ ∈ {-1, +1} and the scale coefficient α ₁ , the product of α ₂ ∈ R to fit the full-precision weights, W ₁ ≈α ₁ B ₁ , W ₂ ≈α ₂ B ₂ .

(2) A necessary constraint is added to the fitting weight problem in (1), that is, α ₁ =α ₂ , which is represented by α uniformly. The purpose is that the result of adding the two parallel weights is a ternary representation, ie α ₁ B ₁ +α ₂ B ₂ =α(B ₁ +B ₂ ) is a ternary representation.

(3), solving the weight fitting problem to which the constraints of (2) are added to binarize the sub-layers in each alienated convolution operation layer. Specifically, it is solved by an optimization problem of minimizing the quantization error between the full-precision weights W ₁ , W ₂ and the binarized weights B ₁ , B ₂ .

使得α₁＝α₂。将所述最优化问题展开得到

其中

是一个不影响优化求解的常数项。由B₁，B₂∈{-1，+1}^nchw可得

是常数项。当二值权重和相应的全精度权重拥有相同的符号时上述最优化问题取得最小值，即

其中，

和

为前文所述两个并列的全精度卷积层，

和

为对应卷积层中第i个卷积核，

和

为前文所述两个并列的二值化卷积层，N为该卷积层中卷积核的个数，α^*为前文所述尺度系数。This optimization problem can be modeled as minimizing the following optimization problem:

Let α ₁ =α ₂ . Expanding the optimization problem, we get

in

is a constant term that does not affect the optimization solution. From B ₁ , B ₂ ∈ {-1, +1} ^nchw can be obtained

is a constant term. The above optimization problem achieves a minimum when the binary weights and the corresponding full-precision weights have the same sign, that is,

in,

and

are the two parallel full-precision convolutional layers described above,

and

is the i-th convolution kernel in the corresponding convolutional layer,

and

is the two parallel binarized convolution layers mentioned above, N is the number of convolution kernels in the convolution layer, and α ^* is the scale coefficient mentioned above.

中sign()函数在位置0处是不可导的，因此在无法直接使用梯度反向传播对二值化的权值进行更新。使用对应的全精度权值的梯度作为二值化权值的梯度的近似，

是全精度权值W的近似，则损失函数l对第i个权值W_i的梯度为

对其中不可导的sign()函数，使用

近似，其中

表示当|W_i|≤1时其值为1，否则为0。将(1)中的两个并列二值权重B₁、B₂代入上式，全精度权值W的近似估计

由于(3)中的尺度系数α同时依赖于

和

因此在计算W₁中第i个卷积核

的梯度时要同时考虑其他卷积核W₁ ^j和

的影响，其中

和W₁ ^j是同一个卷积层中不同的卷积核，

是与

并列卷积层中的卷积核。则损失函数l对权值

的梯度为In this embodiment,

The sign() function is non-derivative at position 0, so the binarized weights cannot be updated directly using gradient backpropagation. Use the gradient of the corresponding full-precision weight as an approximation to the gradient of the binarized weight,

is an approximation of the full-precision weight W, then the gradient of the loss function l to the _ith weight Wi is:

For the non-derivable sign() function, use

approximately, where

Indicates that the value is 1 when |W _i |≤1, and 0 otherwise. Substitute the two parallel binary weights B ₁ and B ₂ in (1) into the above formula, the approximate estimation of the full-precision weight W

Since the scale coefficient α in (3) also depends on

and

Therefore, in calculating the ith convolution kernel in W ₁

The gradients of other convolution kernels W ₁ ^j and

the impact of which

and W ₁ ^j are different convolution kernels in the same convolution layer,

With

Convolution kernels in parallel convolutional layers. Then the loss function l pairs the weights

The gradient of is

分别是(1)的两个并列的卷积核。in

are the two parallel convolution kernels of (1).

2. Network optimization

The intermediate network is trained based on the training data to obtain an optimized intermediate network; a ternary weight is obtained by adding the binarized weights of the two sublayers in each alienated convolution operation layer in the optimized intermediate network , get the ternary convolutional neural network.

In the ternary convolutional neural network of this embodiment, an activation quantization layer Ternarize() is set before each convolution operation layer, and the input X _i of this layer is subjected to ternarization processing:

为三值化处理的输出。in,

is the output of the ternarization process.

The activation values are ternarized so that the convolution operation between the ternary weights and the ternary activations can be replaced by low-energy bit operations.

After the network training is completed, a network model consisting of two parallel binary weights is obtained. Add the two parallel binary weights B ₁ and B ₂ to obtain the ternary weight T, and then perform the convolution operation with the ternary input X to obtain the output Y:

FIG. 4 exemplarily shows the process of adding two-valued convolution kernels represented by two-dimensional convolution to obtain a three-valued convolution kernel. The upper part of Fig. 4 is the process of convolving the two parallel binary weights with the input. The input is convolved with the binary convolution kernels B ₁ and B ₂ respectively, and the intermediate results "Output 1" and "Output 1" are obtained. 2", add the results to get the final output result; the lower part is the process of convolving the three-valued weights obtained by the addition with the input. The results obtained by both are equal. When deploying the network model, only the three-valued convolution kernel in the lower half of Figure 4 needs to be retained.

For convolutional layers and fully-connected layers, the ternary weights and activated convolution operations can be replaced by bit operations, which can significantly reduce the computational overhead and improve the running speed.

In the example of the present invention, the weights and activations of the deep convolutional neural network are ternarized, the weights and activations are converted from 32-bit floating point numbers to 3 integer values, and the weights are stored in a 2-bit format to compress the network model. the goal of. At the same time, the convolution operation is also replaced by the original floating-point multiplication and addition operation, which is replaced by a bit operation, so as to achieve the purpose of accelerating the forward inference speed of the network.

The method provided by the present invention can realize the acceleration and compression of the deep convolutional neural network, and one of its advantages is that it provides a calculation scheme of soft threshold, which is different from the scheme of artificially setting hard threshold in the previous ternary method. In the previous ternarization scheme, a threshold Δ needs to be set manually. For a given full-precision weight X, when -Δ<X<Δ, X is quantized to 0; when X>Δ, X is quantized to 1; when X<-Δ, X is quantified to -1. The artificially set hard threshold Δ in the previous scheme adds extra constraints to the three-valued optimization problem, which is one of the reasons for the low accuracy of its three-valued method. The present invention is no longer constrained by the hard threshold, and can automatically determine which position elements in the three-value quantization are quantized as 0: the corresponding positions in the two parallel convolution kernels are the positions of opposite numbers +1 and -1 respectively, and automatically after the addition Introduce 0. Compared with the previous hard threshold ternarization scheme, the soft threshold ternarization scheme of the present invention has relaxed constraints and a larger search space, so the model accuracy on classification tasks is much higher than the previous hard threshold scheme.

The fully connected layer can be regarded as a special convolutional layer, so the fully connected layer also has the above characteristics. For the convolutional layer and the fully-connected layer, the weights after ternarization are far smaller than the original weights storage size before being processed by the embodiment of the present invention, and the computational complexity of convolution is greatly reduced, so it can be Significantly reduces the storage overhead of convolutional neural network weights and the running time of convolutional neural network, thereby improving the running speed.

The following takes the commonly used ResNet18 as an example to briefly describe the method of the present invention:

Get the ResNet18 deep convolutional neural network applied in image classification;

The ResNet18 deep convolutional neural network is processed by using the method provided by the above-mentioned embodiments of the present invention to obtain a ResNet18 deep convolutional neural network with two parallel convolution kernels in each layer;

In the training process based on gradient backpropagation, binary quantization is performed on the convolutional layers in the ResNet18 deep convolutional neural network with two parallel convolution kernels in each layer;

After the training, the two parallel binary convolution kernels in the above ResNet18 deep convolutional neural network are added to obtain a ternary ResNet18 deep convolutional neural network.

Through testing, the storage space occupied by the tertiary ResNet18 deep convolutional neural network after being processed by the method provided by the embodiment of the present invention is reduced by at least 16 times. The test accuracy on ImageNet, a large-scale image classification task, reaches 66.21%, which is the highest accuracy among the currently known ternary networks.

A convolutional neural network optimization system based on soft threshold ternary parameters according to the second embodiment of the present invention includes an intermediate network module and a network optimization module;

The building intermediate network module is configured to build an intermediate network through the following methods:

Step S100, each convolution operation layer of the original convolutional neural network is divided into two parallel sub-layers with the same convolution kernel size, as the alienated convolution operation layer; the convolution operation layer includes convolution layers and fully connected layers;

Step S200, under the constraint of the same equal-scale coefficients, binarize the sub-layers in each alienated convolution operation layer to obtain an intermediate network;

The network optimization module is configured to train the intermediate network based on the training data, obtain an optimized intermediate network, and binarize the two sublayers in each alienated convolution operation layer in the optimized intermediate network. The weights are added to obtain a three-valued weight to obtain a three-valued convolutional neural network.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process and related description of the system described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here.

It should be noted that the convolutional neural network optimization system based on the soft threshold ternary parameter provided by the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be allocated as required. It is completed by different functional modules, that is, the modules or steps in the embodiments of the present invention are decomposed or combined. For example, the modules in the above embodiments can be combined into one module, and can also be further split into multiple sub-modules to complete the above description. all or part of the functions. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing each module or step, and should not be regarded as an improper limitation of the present invention.

A storage device according to the third embodiment of the present invention stores a plurality of programs, and the programs are adapted to be loaded and executed by a processor to realize the above-mentioned optimization method of a convolutional neural network based on a soft threshold ternary parameter.

A processing device according to a fourth embodiment of the present invention includes a processor and a storage device; the processor is adapted to execute various programs; the storage device is adapted to store multiple programs; the programs are adapted to be loaded and executed by the processor In order to realize the above-mentioned optimization method of convolutional neural network based on soft threshold ternary parameters.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and relevant description of the storage device and processing device described above can refer to the corresponding process in the foregoing method embodiments, which is not repeated here. Repeat.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion, and/or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the method of the present application are performed. It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for performing the operations of the present application may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional Procedural programming language - such as the "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

The terms "first," "second," etc. are used to distinguish between similar objects, and are not used to describe or indicate a particular order or sequence.

The term "comprising" or any other similar term is intended to encompass a non-exclusive inclusion such that a process, method, article or device/means comprising a list of elements includes not only those elements but also other elements not expressly listed, or Also included are elements inherent to these processes, methods, articles or devices/devices.

So far, the technical solutions of the present invention have been described with reference to the preferred embodiments shown in the accompanying drawings, however, those skilled in the art can easily understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims (10)

1. A convolution neural network optimization method based on soft threshold three-valued parameters is characterized by comprising two parts of constructing an intermediate network and optimizing the network:

constructing an intermediate network, comprising the steps of:

step S100, splitting each convolution operation layer of the original convolution neural network into two parallel sub-layers with the same convolution kernel size as a dissimilatory convolution operation layer; the convolution operation layer comprises a convolution layer and a full connection layer;

step S200, carrying out binarization on sub-layers in each dissimilarity convolution operation layer respectively under the constraint condition of the same equal-scale coefficient to obtain an intermediate network;

network optimization, comprising the steps of:

training the intermediate network based on training data to obtain an optimized intermediate network;

and adding the binary weights of the two sub-layers in each dissimilarity convolution operation layer in the optimized intermediate network to obtain a ternary weight, so as to obtain a ternary convolution neural network.

2. The soft-threshold tri-valued parameter-based convolutional neural network optimization method of claim 1, wherein said two sublayers are collocated in position and have the same size as the original convolutional kernel in shape.

3. The soft threshold tri-valued parameter based convolutional neural network optimization method of claim 1, wherein the convolutional operation layer comprises layers 2 to L-1 of an L-layer convolutional neural network; where L is the total number of layers of the convolutional neural network.

4. The soft-threshold tri-valued parameter-based convolutional neural network optimization method of claim 1, wherein the output of the differentiated convolutional operation layer is equal to the input of the layer, and the input of the differentiated convolutional operation layer is respectively convolved with the convolutional kernels of two parallel sublayers and then added.

5. The soft threshold ternary parameter-based convolutional neural network optimization method of claim 1, wherein in step S200, "binarize sub-layer in each dissimilatory convolutional operation layer", the method is as follows:

by minimizing the full-precision weight W₁、W₂And a binarization weight B₁、B₂Solving an optimization problem of quantization error between; wherein, W₁、W₂Full-precision weights, B, for two sublayers in a dissimilarity convolution operation layer₁、B₂Respectively, the binarization weights of two sub-layers in the dissimilarity convolution operation layer.

6. The soft-threshold tri-valued parameter-based convolutional neural network optimization method of claim 5, wherein the constraint condition for solving the optimization problem is that the constraint condition is that the equal-scale coefficients α of the binarization weights of two sub-layers in the dissimilarity convolutional operation layer₁、α₂Are equal.

7. The soft-threshold tri-valued parameter-based convolutional neural network optimization method of claim 1, wherein each convolution operation layer of the tri-valued convolutional neural network is preceded by an activation quantization layer ternizine (), and an input X to the layer is input to the activation quantization layer ternizine ()_iCarrying out ternary processing:

wherein,

is the output of the tri-valued process.

8. A convolution neural network optimization system based on soft threshold three-valued parameters is characterized by comprising a middle network building module and a network optimization module;

the intermediate network construction module is configured to construct an intermediate network by:

step S100, splitting each convolution operation layer of the original convolution neural network into two parallel sub-layers with the same convolution kernel size as a dissimilatory convolution operation layer; the convolution operation layer comprises a convolution layer and a full connection layer;

step S200, carrying out binarization on sub-layers in each dissimilarity convolution operation layer respectively under the constraint condition of the same equal-scale coefficient to obtain an intermediate network;

the network optimization module is configured to train the intermediate network based on training data to obtain an optimized intermediate network, and add the binary weights of the two sublayers in each dissimilarity convolution operation layer in the optimized intermediate network to obtain a ternary weight to obtain a ternary convolution neural network.

9. A storage device having stored therein a plurality of programs, wherein said programs are adapted to be loaded and executed by a processor to implement the soft threshold ternary parameter based convolutional neural network optimization method of claims 1-7.

10. A processing device comprising a processor, a storage device; a processor adapted to execute various programs; a storage device adapted to store a plurality of programs; characterized in that said program is adapted to be loaded and executed by a processor to implement the soft threshold ternary parameter based convolutional neural network optimization method of claims 1-7.

Images