WO2020213670A1 - Neural calculation device and neural calculation method - Google Patents

Abstract

The present invention provides a neural calculation device 1, etc., with which it is possible to dynamically omit calculations relating to a neuron. The present invention comprises: a neural net calculation unit 2 composed of a plurality of neuron calculators 21a that add up the results obtained by multiplying input data i and a weighting coefficient w and applying an activation function to the addition result to output output data o; a zero output specification unit 3 for performing a calculation to specify a neuron calculator 21a for which the output is zero depending on the input data i; and a control unit 4 for controlling the calculation of the neural net calculation unit 2 on the basis of the output of the zero output specification unit 3 so as to omit the calculation of the neuron calculator 21a for which the output is zero.

Description

Neural calculation device and neural calculation method

The present invention belongs to the technical fields of neural computing devices and neural computing methods.

In recent years, there has been a demand for computer hardware that processes deep neural networks (DNN) at high speed and low power consumption for applications such as IoT (Internet of Things) and autonomous driving. Most of the calculation of the neural network is the multiplication of the weighting coefficient and the activity value on the input data and the integration of the multiplication result. Therefore, as a method of reducing the number of multiplications and additions by omitting weights that do not significantly affect the final recognition result, a technique called pruning (for example, Non-Patent Document 1) or a similar technique is used. There is a technique called Neuron Pruning (for example, Non-Patent Document 2) that removes neurons that do not significantly affect the recognition result.

"Learning both Weights and Connections for Efficient Neural Networks" paper, Song Han et al., ArXiv: 1506.02626v3 [cs.NE], October 30, 2015 "A Threshold Neuron Pruning for a Binarized Deep Neural Network on an FPGA" paper, Tomoya Fujii et al., IEICE Transactions on Information and Systems, 2018 E101.D Volume 2 p.376-386 (URL: https: // www. jstage.jst.go.jp/article/transinf/E101.D/2/E101.D_2017RCP0013/_article/-char/ja/)

However, in the conventional technique, since the weighting coefficient and the neuron are statically deleted when the neural network is learned, it is not possible to omit the calculation regarding the invalid weight and the neuron that are dynamically expressed according to the input pattern.

Therefore, the present invention has been made in view of each of the above problems and requirements, and one example of the problem is to provide a neural computing device capable of dynamically omitting calculations related to neurons.

In order to solve the above problem, the invention according to claim 1 adds the multiplication result of the input data and the weighting coefficient, applies an activation function to the addition result, and outputs the output data. A neural net calculation means composed of a plurality of neuron calculators, a zero output specifying means for performing a calculation for specifying the neuron calculator whose output becomes zero according to the input data, and the zero output specifying means. Based on the calculation result of the above, the computer is provided with a control means for controlling the calculation of the neural net calculation means so as to omit the calculation of the neuron calculator whose output becomes zero.

According to the second aspect of the present invention, in the neural calculation apparatus according to the first aspect, the zero output specifying means is a binarized neural network calculator that imitates the calculation of the neural network calculation means, and the control means. However, it is characterized in that the calculation of the neural network calculation means is controlled based on the output of the zero output specifying means in which 1-bit data obtained by converting from the input data is input.

The invention according to claim 3 is a binarization that imitates the calculation before the zero output specifying means applies the activation function to the neural network calculation means in the neural computer according to claim 2. It is characterized by being a neural computer.

The invention according to claim 4 is the binarized neural computer according to claim 3, based on the difference between the output of the zero output specifying means and the addition result in the neural network calculation means in the neural computer according to claim 3. It is characterized by further providing a learning means for learning.

In the invention according to claim 5, in the neural calculation apparatus according to any one of claims 1 to 4, the neural network calculation means is composed of a plurality of neural layers and corresponds to each of the neural layers. There are a plurality of the zero output specifying means, and the zero output specifying means corresponding to the neural layer makes the output of the neuron computer belonging to the neural layer zero according to the output data of the neural layer of the previous layer. It is characterized by performing a calculation that identifies a neural computer.

The invention according to claim 6 is based on the neural computer according to claim 5, wherein the control means is specified by the zero output specifying means corresponding to the neural layer to have a non-zero output. It is characterized in that the calculation of the neural network calculation means is controlled so that the calculation is started in advance.

The invention according to claim 7 is the neural computing device according to claim 5 or 6, wherein the control means has a time interval between the calculation in the neural layer and the calculation in the neural layer of the next layer. It is characterized in that the calculation of the neural network calculation means is controlled so as to reduce the power consumption.

The invention according to claim 8 is characterized in that the activation function is a rectified linear function in the neural computing device according to any one of claims 1 to 7.

In the invention according to claim 9, in the neural computer according to claim 8, the zero output specifying means identifies the neuron computer in which the sign of the addition result is zero or less from the pattern of the input data. It is characterized by performing calculations to be performed.

The invention according to claim 10 is composed of a plurality of neural computers that add a multiplication result of input data and a weighting coefficient, apply an activation function to the addition result, and output output data. In the neural network computing device, the zero output specifying means performs a calculation for specifying the neuron computer whose output becomes zero according to the input data, and the control means is the zero output specifying means. It is characterized by including a control step for controlling the calculation of the neural network calculation device so as to omit the calculation of the neuron computer whose output becomes zero based on the calculation result of.

According to the present invention, a neural network calculation composed of a plurality of neuron calculators that add the multiplication result of input data and weighting coefficient, apply an activation function to the addition result, and output output data. In, a calculation is performed to identify a neuron computer whose output becomes zero according to the input data, and based on this calculation result, the neural network calculation is controlled so as to omit the calculation of the neuron computer whose output becomes zero. By doing so, it is possible to identify the neural computer whose output becomes zero according to the pattern of the input data, so that the calculation related to the neural network can be dynamically omitted.

It is a block diagram which shows an example of the neural calculation apparatus which concerns on embodiment. It is a block diagram which shows the outline structure example of the neural network calculation part and zero output specific part of FIG. It is a block diagram which shows an example of the neuron calculator of FIG. It is a schematic diagram which shows an example which omitted the calculation of a neuron calculator. It is a block diagram which shows the outline structure example of the binarized neural network system. It is a block diagram which shows an example of the neural electronic circuit of FIG. It is a flowchart which shows the operation example of the neural calculation apparatus. It is a block diagram which shows an example of a neural network calculation part and zero output specific part. It is a schematic diagram which shows an example of the operation timing of the neural network calculation part and zero output specific part. It is a schematic diagram which shows an example of the operation timing of a neural network calculation part and a zero output specific part. It is a schematic diagram which shows an example of the result of the activity prediction accuracy by a neural calculation apparatus. It is a schematic diagram which shows an example of the result of the calculation reduction rate by a neural calculation apparatus. It is a block diagram which shows the modification of the zero output specific part. The zero output identification part is the schematic diagram explaining the identification function of a support vector machine. It is a block diagram which shows the modification of the neural network calculation part and the zero output specific part.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiment described below is an embodiment when the present invention is applied to a neural computing device.

(1. Outline of configuration and function of neural computing device)
First, the configuration and outline function of the neural computing device according to the embodiment of the present invention will be described with reference to FIG.

FIG. 1 is a block diagram showing an example of a neural computing device according to an embodiment. FIG. 2 is a block diagram showing an outline configuration example of the neural network calculation unit and the zero output specific unit. FIG. 3 is a block diagram showing an example of a neuron calculator. FIG. 4 is a schematic diagram showing an example in which the calculation of the neuron calculator is omitted.

As shown in FIG. 1, the neural calculation device 1 adds the multiplication result of the input data i and the weighting coefficient, applies an activation function to the addition result, and outputs the output data o. The neural network calculation unit (NN calculation unit) 2 composed of the instruments, the zero output specification unit 3 that performs the calculation to specify the neural network calculation unit whose output of the neural network calculation unit 2 becomes zero, and the zero output Based on the specific unit 3, the control unit 4 that controls the calculation of the neural network calculation unit 2 so as to omit the calculation of the neuron computer whose output becomes zero, and the memory 5 that stores the weighting coefficient w and the intermediate data of the calculation. And have.

The neural network calculation unit 2 is a neural network composed of a plurality of neuron calculators that add the multiplication result of the input data and the weighting coefficient, apply an activation function to the addition result, and output the output data. This is an example of a calculation means. The zero output specifying unit 3 is an example of a zero output specifying means that performs a calculation for specifying the neuron computer whose output becomes zero according to the input data. The control unit 4 is an example of a control means that controls the calculation of the neural network calculation means so as to omit the calculation of the neuron computer whose output becomes zero based on the calculation result of the zero output specifying means.

Neural computing device 1, the input data _{i 0,} the input data i _1, the input data i _2, ..., the input data i _n (n is a natural number. Hereinafter the same.) With respect to the output data _{o 0,} the output data o _1, output data o _2, ..., input and output data o _m (m is a natural number. forth.) to the.

As shown in FIG. 2, the neural network calculation unit 2 is composed of a multi-layer neural layer 20 having a plurality of neuron calculators 20a.

As shown in FIG. 3, the neuron calculator 20a is input data _{i 0,} the input data i _1, the input data i _2, ..., and input data i _n, the weighting factor _{w 0} for each of the input data i, the weighting factor w _It has a multiplication adder 21a that adds a multiplication result of ₁ , a weighting coefficient w ₂ , ..., A weighting coefficient w _n , and an activation function device 25a that applies an activation function to the addition result. When the activation function of the activation function device 25a is a rectified linear function, if the output of the multiplication adder 21a is zero or less, zero is output, and if the output of the multiplication adder 21a is a positive value x, the value. Output x.

Note that, in FIG. 2, each layer of the neural network is schematically drawn with the output side neuron and the input side neuron separated between the adjacent neural layers 20. In the first neural layer 20, the neuron calculator 20a group of the first layer L1 is configured with respect to the neurons of the input layer L0. In the second neural layer 20, the neuron calculator 20a group of the second layer L2 is configured with respect to the neurons of the first layer L1. In the nth neural layer 20, a neuron calculator 20a group of the nth layer Ln is configured with respect to the neurons of the n-1th layer Ln-1.

As shown in FIG. 2, the zero output specifying unit 3 is composed of a plurality of activity predictors 30. Each activity predictor 30 is installed corresponding to each neural layer 20.

The same input data as the corresponding neural layer 20 is input to the activity predictor 30. The output of the neural layer 20 of the previous layer is input to the activity predictor 30 corresponding to the neural layers 20 of the second and subsequent layers.

The activity predictor 30 performs a calculation imitating the calculation before applying the activation function among the calculations in each neuron computer 20a of the neural layer 20 based on the same input data pattern as the corresponding neural layer 20. The calculation result in the activity predictor 30 is output to the control unit 4.

In contrast to the neurocomputer 20a whose output is predicted to be zero or less based on this calculation result, the control unit 4 omits the calculation of multiplication and addition related to the neurocomputer 20a whose output becomes zero. The calculation of the neural layer 20 is controlled so as to.

For example, as shown in FIG. 4, the neural network calculation unit 2 calculates only the neuron calculator 20a connected by the network of the neuron calculator 20a without omitting the calculation.

The activity predictor 30 may be a computer that can calculate faster than the neural layer 20. For example, as an example of the activity predictor 30, a binarized neural computer, an integerized neural computer, a support vector machine (SVM), a random forest, and the like can be mentioned. When the activity predictor 30 is a binarized neural computer, the activity predictor 30 takes out, for example, the most significant bit of the input data on the input side, and calculates the binarized neural network. In the first layer, the most significant bit represents the sign of the input data (eg, 0 is a minus sign and 1 is a plus sign), and in the second and subsequent layers, the most significant bit is the magnitude of the data value (eg,). , 0 indicates 0, and 1 indicates the maximum value that can be indicated by the number of bits). The input used for the calculation of the binarized neural network may be 1-bit data obtained by converting from the input data in addition to the most significant bit of the input data.

The calculation of the neural net calculation unit 2 and the zero output specific unit 3 may be performed by the hardware of a dedicated electronic circuit, by a CPU such as a Neumann type, or by an AI chip such as a GPU. However, the calculation may be a combination of the CPU and the GPU, or may be a calculation using a neuromorphic chip.

The calculation of the binarized neural network in the zero output specifying unit 3 is performed by, for example, a neural electronic circuit that performs binarization multiplication by the exclusive OR (XNOR) of the 1-bit weighting coefficient and the 1-bit input data. You may. The calculation of the binarized neural network adds to the multiplication result of the binarized multiplication based on the 1-bit connection presence / absence information between neurons. The calculation of the binarized neural network applies an activation function to the addition result and outputs 1-bit output data.

(2. Configuration and function of binarized neural network system in zero output identification unit 3)
Next, the configuration and outline function of the binarized neural network system when the zero output specifying unit 3 is the binarized neural network will be described with reference to the drawings.

FIG. 5 is a block diagram showing an outline configuration example of a binarized neural network system. FIG. 6 is a block diagram showing an example of the neural electronic circuit of FIG.

As shown in FIG. 5, the binarized neural network system NNS includes a plurality of core electronic circuit Cores capable of realizing various types of neural networks with electronic circuits, and a system bus bus connecting the core electronic circuit Cores. , Is equipped.

The core electronic circuit Core includes a binarized neural electronic circuit NN that can realize various types of neural networks with an electronic circuit, a memory access control unit MCnt that sets weighting coefficients of the binarized neural electronic circuit NN, and a neural network. It has an electronic circuit NN and a control unit Cnt that controls a memory access control unit MCnt. Here, as an example of various types of neural networks, a fully connected type neural network in which neurons between neurons are fully connected to each other, a neural network that performs convolution calculation, a neural network that is expanded in the neuron layer, and a layer. Examples include neural networks that expand numbers.

The binarized neural electronic circuit NN has an input memory array unit MAi that sequentially supplies input data I1, ..., Ip (p is a natural number; the same applies hereinafter) in parallel, and a memory that sequentially supplies weighting coefficient data in parallel. A cell unit MC, a plurality of process element units Pe that realize a multiplication function for multiplying the supplied input data I1, ..., Ip and a weighting coefficient and output a multiplication result, and a process of each input data in parallel. Addition activation unit Act that adds each multiplication result from element unit Pe and applies an activation function to the addition result, and 1-bit output data O1, ..., Oq (q) from each addition activation unit Act. Has an output memory array unit MAo that sequentially stores natural numbers (the same applies hereinafter), and a bias memory array unit MAb that sequentially provides bias values to each addition activation unit Act.

The memory access control unit MCnt is, for example, a Direct Memory Access Controller. The memory access control unit MCnt sets the input data to be sequentially supplied to each process element unit Pe in the input memory array unit MAi according to the control of the control unit Cnt. Further, the memory access control unit MCnt sets in advance a weighting coefficient and a predetermined value indicating the presence / absence of connection between neurons in each memory cell unit MC according to the control of the control unit Cnt. Further, the memory access control unit MCnt takes out the output data output from the addition activation unit Act from the output memory array unit MAo under the control of the control unit Cnt.

The control unit Cnt has a CPU (Central Processing Unit) and the like. The control unit Cnt measures timing such as synchronization of each element of the binarized neural electronic circuit NN, and synchronizes calculation and data transfer. In addition, the control unit Cnt controls switching of the selector element described later in the binarized neural electronic circuit NN.

The control unit Cnt controls the memory access control unit MCnt, prepares the data output from the other core electronic circuit Core for the input memory array unit MAi, and supplies the data as input data to the input memory array unit MAi. Control. The control unit Cnt controls the memory access control unit MCnt to transfer the output data acquired from the output memory array unit MAo to another core electronic circuit Core.

Note that the host controller (for example, control unit 4) may control the neural network system NNS and the control unit Cnt of each core electronic circuit Core. Further, the host controller may control the binarized neural electronic circuit NN and the memory access control unit MCnt instead of the control unit Cnt. The host controller may be an external computer.

The bias memory array unit MAb stores bias data provided to each addition activation unit Act in advance.

As shown in FIG. 6, the binarized neural electronic circuit NN realizes, for example, a two-layer neural network having p inputs and q outputs.

The memory cell unit MC has a memory cell 10 that stores a weighting coefficient. The memory cell 10 stores a preset 1-bit weighting coefficient of “1” or “0” based on the brain function realized by the neural network to be constructed.

The memory cell unit MC may also have another memory cell (not shown) for connection presence / absence information that stores connection presence / absence information between neurons preset based on the brain function. .. Here, the no-connection information is, for example, a 1-bit predetermined value meaning NC (Not Connected), and "1" or "0" or the like is assigned as the predetermined value.

Memory cells 10 are lined up to form a row of memory cells. At the same time, the memory cell block CB is formed by collectively forming the memory cells 10 output to each process element unit Pe. The memory cell 10 of the memory cell block CB corresponds to each input data input in parallel.

The memory cell unit MC preferably has a memory cell block CB having an input parallel number of p or more of input data I1, ..., Ip input in parallel from the input memory array unit MAi. In the memory cell block CB, the number of memory cells 10 is preferably equal to or greater than the number of cycles of serial input data sequentially input by 1 bit from the input memory array unit MAi.

The memory cell unit MC sequentially outputs a 1-bit weighting coefficient for each memory cell block CB to the process element unit Pe corresponding to the serial input data input sequentially by 1 bit. The weighting coefficient from the memory cell block CB and the input data from the input memory array unit MAi are input to each process element unit Pe.

The memory cell block CB may alternately and sequentially output a 1-bit weighting coefficient and a 1-bit connection presence / absence information to the process element unit Pe. The memory cell 10 has an independent connection to the process element unit Pe, and may be sequentially output to the process element unit Pe separately.

As shown in FIGS. 5 and 6, the memory cell unit MC has q output parallel outputs of output data output in parallel to the output memory array unit MAo, output data O1, ..., Oq output in parallel. Correspondingly, it is arranged in the binarized neural electronic circuit NN.

As shown in FIG. 6, the process element unit Pe of the number of input parallel ps arranged in each parallel input data input in parallel is a process element column (for example, a process) in the binarized neural electronic circuit NN. Form the element column PC1). The process element columns PC1 to PCq having a number of output parallels of q are arranged in the q column in the binarized neural electronic circuit NN corresponding to the output data output in parallel. As shown in FIG. 3, the process element unit Pe is set as a two-dimensional arithmetic unit array in p rows × q columns in the binarized neural electronic circuit NN.

Input data I1 is commonly input to the process element part Pe of the matrix (1,1), (1,2), ..., (1,q). Input data I2 is commonly input to the process element parts Pe of the matrices (2,1), (2,2), ..., (2, q). Input data Ip is commonly input to the process element section Pe of the matrix (p, 1), (p, 2), ..., (P, q).

The process element unit Pe calculates and outputs the exclusive OR (XNOR) of the 1-bit weighting coefficient output from the corresponding memory cell 10 and the 1-bit input data as the multiplication result and outputs it.

Note that when no connection information (for example, a predetermined value meaning "NC") is output from the memory cell for connection presence / absence information, the multiplication result is not added in the addition activation unit Act. For example, the multiplication result and the connection presence / absence information may be alternately output as a pair. Further, regarding the connection presence / absence information, the process element unit Pe may have a connection independent of the multiplication result from the addition activation unit Act, and the multiplication result and the connection presence / absence information may be output separately.

When calculating the partial sum of the multiplication results in the process element unit Pe, if no connection information (for example, a predetermined value meaning "NC") is output from the memory cell for connection presence / absence information, the multiplication result Is not added to the partial sum of.

The process element columns PC1, ..., PCq output the multiplication result from each process element unit Pe or the partial sum result obtained by adding a part of the multiplication results to the addition activation unit Act.

As shown in FIGS. 5 and 6, the addition activation unit Act is arranged according to each output data O1, ..., Oq output in parallel.

The addition activation unit Act adds the multiplication results sequentially output from the process element column based on the connection presence / absence information, applies an activation function to the addition result, and outputs 1-bit output data. Memory array unit MAo Output to. When the process element unit Pe outputs the partial sum of the multiplication results, the addition activation unit Act adds the multiplication results sequentially output from the process element column, applies an activation function to the addition result, and 1 bit. Outputs the output data of to the output memory array unit MAo.

In the process element column, the addition activation unit Act determines in advance a value obtained by subtracting the number of times "0" is calculated as the multiplication result from the number of times "1" is calculated as the multiplication result in one cycle unit of the input data. When it is equal to or more than the specified threshold value, "1" is output as output data, and when the subtracted value is less than the threshold value, "0" is output as output data.

As shown in FIG. 6, the bit serial input is parallelized, the row of the process element part Pe is shared with respect to the input data, and each process element column which is a column of the process element part Pe is output independently. Output data.

Note that the binarized neural electronic circuit NN may have a logarithmic circuit so that it can handle multi-bit data in addition to 2-bit data. For example, the binarized neural electronic circuit NN has a circuit that linearizes the multiplication result of the input data and the weighting coefficient by logarithmic addition of the logarithmic input data and the logarithmic weighting coefficient by inverse conversion. May be good. Further, the zero output specifying unit 3 may be a machine learning method that finally returns 0/1 in addition to the binarized neural electronic circuit NN.

(3. Operation example of neural computing device)
Next, an operation example of the neural calculation device 1 will be described with reference to FIGS. 7 to 12. The operation will be described by taking as an example a case where the neural calculation device 1 is a convolutional neural network and the zero output specifying unit 3 is a binarized neural network.

FIG. 7 is a flowchart showing an operation example of the neural computing device. FIG. 8 is a block diagram showing an example of a neural network calculation unit and a zero output specific unit. 9 and 10 are schematic views showing an example of the operation timing of the neural network calculation unit and the zero output specific unit. FIG. 11 is a schematic diagram showing an example of the result of the activity prediction accuracy by the neural computing device. FIG. 12 is a schematic diagram showing an example of the result of the calculation reduction rate by the neural computing device.

As shown in FIG. 7, the neural computing device 1 acquires the trained neural network model (step S1). Specifically, the control unit 4 of the neural calculation device 1 acquires the value of the weighting coefficient necessary for constructing the convolution neural network from the memory 5. The control unit 4 outputs the value of the weighting coefficient to the neural network calculation unit 2 and sets the trained neural network model. Further, the control unit 4 sets the activation function as a rectified linear function.

For example, as shown in FIG. 8, in the neural network calculation unit 2, when the neural layer 20 is a convolution layer, a multiplication addition unit 21 that performs a convolution operation and a rectified linear function unit 25 are set. The neural layer 20 of the output layer may be a fully connected layer. The neural layer 20 of the hidden layer may include a fully connected layer.

The trained neural network model is constructed by pre-learning the neural network calculation unit 2 with a predetermined learning data set. For example, a data set for learning is applied to the neural network calculation unit 2 and learning is performed by the error propagation method.

Next, the neural calculation device 1 trains the zero output specifying unit 3 (step S2). For example, as shown in FIG. 8, when the activity predictor 30 is a binarized neural computer 31, the control unit 4 is connected to each binarized neural computer 31 of the zero output specifying unit 3 and the neural network calculation unit 2. The zero output specific unit 3 is trained by applying the same training data set applied to. More specifically, the control unit 4 has an OFM (Output Feature Map) which is the output of the multiplication / addition unit 21 in each neural layer 20 and an OFM which is the output of the binarized neural computer 31 corresponding to the neural layer 20. And get.

The control unit 4 uses the OFM of the multiplication / addition unit 21 as a teacher signal to obtain an error from the OFM of the binarized neural computer 31, and trains each binarized neural computer 31 by an error propagation method based on this error. .. The control unit 4 corrects the weighting of the binarized neural computer 31 by an optimization method such as Newton's method or the steepest descent method until the value of the loss function becomes a predetermined value or less as an error, and repeats the learning. The error is, for example, the square of the difference between the output of the binarized neural computer 31 and the addition result in the multiplication / addition unit 21. By applying the data set for learning, the weighting coefficient for the binarized neural computer 31 is modified to form the binarized neural computer 31 imitating the multiplication / addition unit 21.

Here, the binarized neural computer 31 is realized by, for example, a binarized neural electronic circuit NN. Further, the zero output specifying unit 3 may be realized by a plurality of core electronic circuits Core.

The weighting coefficient for the binarized neural computer 31 is stored in the memory 5 and is corrected every time learning progresses. The control unit 4 learns by sequentially rewriting the weighting coefficient on the memory 5. The modified weighting coefficient is transferred from the memory 5 to the memory access control unit MCnt of the zero output specific unit 3, and is set in the memory cell unit MC.

When performing the convolution operation, in the binarized neural electronic circuit NN, the input data in the input memory array unit MAi and the weighting coefficient sequentially output from the memory cell unit MC to the process element unit Pe are performed in the convolution operation. The convolution operation is realized under the control of. In the case of an RGB image, the input data is p = 3. When the input image is k × k pixels, the input data i1, i2, ..., Ik, ..., Ik ² are sequentially input. The value of q is the number of neurons in the next layer.

The learned activity predictor 30 or the binarized neural network computer 31 is an example of the binarized neural network computer that imitates the calculation of the neural network calculation means. The learned activity predictor 30 or the binarization neural computer 31 is a binarization neural computer that imitates the calculation (calculation of the multiplication / addition unit 21) before applying the activation function in the neural network calculation means. This is an example. The control unit 4 functions as an example of the learning means for training the binarized neural computer based on the difference between the output of the zero output specifying means and the addition result in the neural network calculation means.

Next, the neural calculation device 1 executes the neural network calculation using the zero output specifying unit 3 (step S3). The control unit 4 reads the learned weighting coefficient for the neural network calculation unit 2 from the memory 5 and loads it into the neural network calculation unit 2. The control unit 4 reads the learned weighting coefficient for the zero output specifying unit 3 from the memory 5 and loads it into the zero output specifying unit 3. In the case of the binarized neural electronic circuit NN, the learned weighting coefficient for the zero output specifying unit 3 is set in the memory cell unit MC corresponding to each binarized neural computer 31.

Next, as shown in FIG. 9, the calculation of the binarized neural computer 31 corresponding to the first neural layer 20 is started. Input data is sequentially input to the first binarization neural computer 31 as a pattern of input data, and binarized by the uppermost bit of the input data (for example, 0 indicates a minus sign and 1 is a plus sign). Calculations are performed on the neural computer 31. As the calculation result of the binarized neural computer 31, the output data O1, O2 ... Of the addition activation unit Act are transmitted from the first binarized neural computer 31 to the control unit 4.

If the output data O1 is "0", the control unit 4 controls the first multiplication / addition unit 21 so as to omit the calculation of the multiplication adder 21a of the neurocomputer 20a corresponding to the output data O1. If the output data O1 is "1", the control unit 4 performs the first multiplication addition so that the calculation of the multiplication adder 21a of the neurocomputer 20a corresponding to the output data O1 is started with the same input data. The unit 21 is controlled. The calculation result of the multiplication / addition unit 21 is output by applying the normalization linear function unit 25.

As described above, the activity predictor 30 or the binarized neural computer 31 functions as an example of the zero output specifying means that performs the calculation for specifying the neuron computer whose output becomes zero according to the input data. To do.

Next, if the output data O2 is "0", the control unit 4 controls the first multiplication / addition unit 21 so as to omit the calculation of the multiplication / addition unit 21a of the neurocomputer 20a corresponding to the output data O2. To do. If the output data O2 is "1", the control unit 4 performs the first multiplication addition so that the calculation of the multiplication adder 21a of the neurocomputer 20a corresponding to the output data O2 is started with the same input data. The unit 21 is controlled. The control unit 4 controls the first multiplication / addition unit 21 for each output data. For example, as shown in FIG. 4, in the first layer L1, the calculation of the multiplication adder 21a whose output is not expected to be zero is performed.

The output from the first multiplication / addition unit 21 and the rectified linear function unit 25 is sequentially input to the second binarization neural computer 31 as a pattern of input data, and the highest bit of the input data (for example, if it is 0). If 0 is indicated, 1 indicates the maximum value that can be indicated by the number of bits), the calculation in the binarized neural network 31 is performed.

Note that the output from the first multiplication / addition unit 21 and the normalized linear function unit 25 may be stored in the cache or the memory 5 of the neural network calculation unit 2 as intermediate data.

The output data of the second binarized neural calculator 31 is sequentially output, and the calculation of the multiplication adder 21a of the corresponding neuron calculator 20a is omitted according to the value of each output (“0” or “1”). Or, the control unit 4 controls the second multiplication / addition unit 21 so as to perform the calculation of the multiplication / adder 21a. For example, as shown in FIG. 4, in the second layer L1, the calculation of the multiplication adder 21a whose output is not expected to be zero is performed.

The outputs from the second multiplication / addition unit 21 and the normalized linear function unit 25 are sequentially input to the third binarization neural computer 31 as input data, and the highest bit of the input data (for example, 0 if 0). , And 1 indicates the maximum value that can be indicated by the number of bits), so that the calculation in the binarized neural network 31 is performed.

The calculation is performed up to the last neural layer 20, and the neural network calculation unit 2 outputs the output data.

As shown in FIG. 9, by omitting the calculation of the multiplication adder 21a of the neuron calculator 20a whose output is predicted to be zero, the calculation can be performed faster than the conventional neural net calculation unit. Whereas the conventional neural net calculation unit has a calculation time t0 in a certain layer, the multiplication / addition unit 21 of the neural net calculation unit 2 calculates at the calculation time t2 (<t0), and the binary value of the zero output specific unit 3 The computerized neural computer 31 calculates at the calculation time t1 (<t0), and the binarized neural computer 31 and the multiplication / addition unit 21 are slightly staggered from each other, so that the calculation is possible from the neuron computer 20a. By processing in parallel to start the calculation, the calculation can be performed in the calculation time t3 (<t0).

The control unit 4 controls the calculation of the neural network calculation means so that the calculation is started in advance from the neuron calculator whose output is specified to be non-zero by the zero output specifying means corresponding to the neural layer. It functions as an example of control means.

As shown in FIG. 10, the calculation time interval between the multiplication / addition unit 21 in the neural network calculation unit 2 and the multiplication / addition unit 21 in the next layer may be separated. After the calculation of the binarized neural computer 31 of the zero output specifying unit 3 is completed, the calculation of the multiplication / addition unit 21 of the next layer may be started. Further, there may be a time when the multiplication / addition unit 21 of the same layer and the binarization neural computer 31 overlap each other.

The control unit 4 controls the calculation of the neural network calculation means so as to reduce the power consumption by providing a time interval between the calculation in the neural layer and the calculation in the neural layer of the next layer. Works as an example.

As described above, the control unit 4 is an example of the control means that controls the calculation of the neural network calculation means based on the output of the zero output specifying means that inputs the 1-bit data obtained by converting from the input data. is there. A calculation in which the binarized neural computer 31 corresponding to the neural layer 20 identifies a neuron computer in which the output of the neuron computer belonging to the neural layer becomes zero according to the output data of the neural layer of the previous layer. This is an example of a zero output specifying means for performing the above. The activity predictor 30 or the binarized neural computer 31 functions as an example of a zero output specifying means that performs a calculation for specifying the neuron computer whose sign of the addition result is zero or less from the pattern of the input data. ..

Next, the results of the simulation will be described with reference to FIGS. 11 and 12. The simulation was performed with a model of 8 convolutional layers, 2 fully connected layers, and 11 layers including the input layer.

As shown in FIG. 11, the activity prediction accuracy of each binarized neural computer 31 tended to decrease as the layers advanced before learning (broken line), but after learning, the activity prediction accuracy of the third layer (dashed line). After conv2_1), the activity prediction accuracy has improved. The horizontal axis is each layer, and the vertical axis is the activity prediction accuracy of each layer. The activity prediction accuracy of each layer is determined by using the number of neuron calculators 20a predicted to be zero as the denominator of the activity prediction accuracy when the correct answer is the case where the calculation is performed by the neural net calculation unit 2 without using the zero output specific unit 3. , The value obtained by using the number of the neurocomputer 20a for which the prediction was correct as the molecule of the activity prediction accuracy.

As shown in FIG. 12, regarding the calculation reduction rate, the calculation reduction rate improved in the third layer (conv2_1) and thereafter. The calculation reduction rate of each layer is a value obtained by using the number of neurocomputers 20a of each layer as the denominator of the calculation reduction rate and the number of neurocomputers 20a of each layer predicted to be zero by the binarized neural computer 31.

From these simulation results, the zero output specifying unit 3 may omit the activity predictor 30 corresponding to the first layer L1 and the activity predictor 30 corresponding to the second layer L2.

As described above, according to the present embodiment, a plurality of devices that add the multiplication result of the input data i and the weighting coefficient w, apply an activation function to the addition result, and output the output data o. In the neural network calculation of the neural network device 1 composed of the neural computer 20a, the zero output specifying unit 3 performs a calculation for specifying the neuron computer 20a whose output becomes zero according to the input data i, and the control unit. Based on this calculation result, 4 controls the neural network calculation of the neural network calculation unit 2 so as to omit the calculation of the neuron computer 20a whose output becomes zero, so that the output is output according to the pattern of the input data. Since the neural computer 20a that becomes zero can be specified, the calculation related to the neural network can be dynamically omitted.

The zero output specifying unit 3 is a binarized neural computer 31 that imitates the calculation of the neural network calculation unit 2, and the control unit 4 is a neural based on the output of the output specifying unit 3 that has input the highest bit of the input data. When controlling the calculation of the net calculation unit 2, the calculation can be performed faster than the neural network calculation means by binarizing the calculation. Therefore, before the neural network calculation unit 2 is calculated, the output becomes zero. Can be identified.

When the zero output specifying unit 3 is a binarized neural computer 31 that imitates the calculation before applying the activation function in the neural network calculation unit 2, the calculation to apply the activation function is omitted and the binary value is obtained. Since the modified neural computer 31 is applied, it is possible to identify the neural computer 20a whose output becomes zero faster.

When the binarized neural computer 31 is trained based on the difference between the output of the zero output specific unit 3 and the addition result in the neural network calculation unit 2, the prediction accuracy of the zero output specific unit 3 is improved by training. ..

The neural network calculation unit 2 is composed of a plurality of neural layers 20, there are a plurality of activity predictors 30 corresponding to each neural layer 20, and the activity predictor 30 corresponding to the neural layer 20 is a neural layer of the previous layer. When a calculation is performed to specify the neuron calculator 20a in which the output of the neuron calculator 20a belonging to the neural layer 20 becomes zero according to the output data of 20, the activity predictor 30 corresponding to each layer has an output of zero. The neural computer 20a can be identified more accurately.

The control unit 4 controls the calculation of the neural network calculation unit 2 so that the calculation is started in advance from the neuron computer 20a whose output is specified to be non-zero by the zero output identification unit 3 corresponding to the neural layer 20. When this is done, the calculation of the neural computer 1 can be realized faster.

When the control unit 4 controls the calculation of the neural network calculation unit 2 so as to reduce the power consumption by providing a time interval between the calculation in the neural layer 20 and the calculation in the neural layer 20 of the next layer. It is possible to save labor in the neural network calculation unit 2 having high power consumption, and to save the energy consumption of the entire neural calculation device 1.

When the activation function is a rectified linear function, if it is zero or less in the application of the rectified linear function, the output is zero, so it is easy to omit the calculation of the neural network calculation unit 2.

When the zero output specifying unit 3 performs a calculation to specify the neuron computer 20a whose sign of the addition result is zero or less from the pattern of the input data, it becomes easier to specify the neuron computer 20a whose output becomes zero.

(4. Modification example)
Next, a modified example of the zero output specific unit will be described with reference to the figure.
FIG. 13 is a block diagram when the zero output specifying unit is a support vector machine. FIG. 14 is a schematic diagram in which the zero output identification unit explains the identification function of the support vector machine and the like. FIG. 15 is a block diagram showing a modified example of the neural network calculation unit and the zero output specific unit.

As shown in FIG. 13, the activity predictor 30 may be a support vector machine classifier 32. The neural calculation device 1 has a neural network calculation unit 2 and a zero output specific unit 3A. The zero output identification unit 3A has a plurality of support vector machine classifiers 32 instead of the binarized neural computer 31. The support vector machine classifier 32 is installed corresponding to the neural layer 20 of each layer.

In step S2, when the activity predictor 30 is the support vector machine classifier 32, the control unit 4 applies the same learning applied to each support vector machine classifier 32 of the zero output specifying unit 3A to the neural network calculation unit 2. The zero output specific unit 3A is trained by applying the data set for. More specifically, the control unit 4 acquires the OFM which is the output of the multiplication / addition unit 21 in the neural layer 20 and the OFM which is the output of the support vector machine classifier 32 corresponding to the neural layer 20.

As shown in FIG. 14, the control unit 4 obtains a difference from the OFM of the support vector machine classifier 32 with the OFM of the multiplication / addition unit 21 as the correct answer label t, and based on the loss function due to this difference, the constrained Newton method or Each support vector machine classifier 32 is trained by an optimization method such as the steepest descent method. The control unit 4 repeats learning until the loss function becomes equal to or less than a predetermined value. By applying the training data set, the weighting coefficient for the support vector machine classifier 32 is modified to form the support vector machine classifier 32 that imitates the multiplication / addition unit 21. As shown in FIG. 14, the identification function of the support vector machine is y = w ^(sum) i + w ₀ ^(sum) .

When the learning of the support vector machine classifier 32 is completed, the neural computer device 1 executes the neural network calculation using the support vector machine classifier 32 instead of the binarized neural computer 31 as in step S3. The support vector machine classifier 32 may be input data that is not converted to the most significant bit, or input data that is binarized to the maximum bit, such as the binarized neural computer 31.

Next, as shown in FIG. 15, the present invention can be applied to a fully bonded layer instead of a convolutional layer. That is, when the neural layer 20 is a fully coupled layer, the activity predictor 30 may be a fully coupled binarized neural computer 33. The multiplication / addition unit 22 of the neural layer 20 full coupling of each layer and the binarized neural computer 33 of the full coupling in the zero output specifying unit 3B correspond to each other.

In the case of full coupling, in the binarized neural electronic circuit NN, the input data in the input memory array unit MAi and the weighting coefficient sequentially output from the memory cell unit MC to the process element unit Pe are calculated for full coupling. It is controlled so that the operation of full coupling is realized. For example, as shown in FIG. 6, in the case of a fully coupled neural network of p × q, a process element column in which process element portions Pe with a number of input parallels p are arranged and a process element column with a number of output parallels q It has PC ₁ , PC ₂ , ..., PC _q, and a memory cell unit MC having a number of output parallels q. The control unit Cnt controls the process element columns PC ₁ , PC ₂ , ..., PC _q and q memory cell units MC of the neural electronic circuit NN to be used. Further, in the case of p × q or more, the core electronic circuits Core may be connected in series or in parallel for realization.

The binarized neural computer 31 that performs the convolution operation and the fully-coupled binarized neural computer 33 have the same configuration, and in performing the operation, they differ only in the weighting and the like supplied from the memory cell unit MC. Therefore, the operation of the binarized neural computer 31 and the binarized neural computer 33 is basically the same.

As described above, whether the activity predictor 30 is the support vector machine classifier 32 or the binarized neural computer 33, the same effect as that of the binarized neural computer 31 can be obtained.

1: Neural calculation device 2: Neural net calculation unit (neural net calculation means)
3, 3A, 3B: Zero output specifying unit (zero output specifying means)
4: Control unit (control means)
20: Neural layer (neural network calculation means)
20a: Neuron calculator 21, 22: Multiplication and addition part 25: Rectifier linear function part 30: Activity predictor (zero output specifying means)
31, 33: Binarized neural computer (zero output identification means)
32: Support vector machine classifier (zero output identification means)
i: Input data w: Weighting coefficient o: Output data

Claims (10)

A neural network calculation means composed of a plurality of neuron calculators that add the multiplication result of the input data and the weighting coefficient, apply an activation function to the addition result, and output the output data.
A zero output specifying means that performs a calculation for specifying the neuron computer whose output becomes zero according to the input data.
A control means that controls the calculation of the neural network calculation means so as to omit the calculation of the neuron computer whose output becomes zero based on the calculation result of the zero output specifying means.
A neural computing device characterized by being equipped with. In the neural computing device according to claim 1,
The zero output specifying means is a binarized neural network computer that imitates the calculation of the neural network calculation means.
A neural calculation apparatus, characterized in that the control means controls the calculation of the neural network calculation means based on the output of the zero output specifying means in which 1-bit data obtained by converting from the input data is input. In the neural computing device according to claim 2,
A neural computing device, characterized in that the zero output specifying means is a binarized neural computer that imitates the calculation before applying the activation function in the neural network calculating means. In the neural computing device according to claim 3,
A neural calculation apparatus further provided with a learning means for training the binarized neural computer based on a difference between the output of the zero output specifying means and the addition result in the neural network calculation means. In the neural computing device according to any one of claims 1 to 4.
The neural network calculation means is composed of a plurality of neural layers.
There are a plurality of said zero output identifying means corresponding to each said neural layer.
The zero output specifying means corresponding to the neural layer performs a calculation to specify a neuron computer in which the output of the neuron computer belonging to the neural layer becomes zero according to the output data of the neural layer of the previous layer. A neural computing device characterized by. In the neural computing device according to claim 5,
The control means controls the calculation of the neural network calculation means so that the calculation is started in advance from the neuron computer whose output is specified to be non-zero by the zero output specifying means corresponding to the neural layer. A neural computing device characterized by this. In the neural computing device according to claim 5 or 6.
The control means controls the calculation of the neural network calculation means so as to reduce power consumption by providing a time interval between the calculation in the neural layer and the calculation in the neural layer of the next layer. Neural computing device. In the neural computing device according to any one of claims 1 to 7.
A neural computing device characterized in that the activation function is a normalized linear function. In the neural computing device according to claim 8,
A neural computing device, characterized in that the zero output specifying means performs a calculation for identifying the neuron computer whose sign of the addition result is zero or less from the pattern of the input data. In a neural network computing device composed of a plurality of neuron calculators that add the multiplication result of input data and weighting coefficient, apply an activation function to the addition result, and output output data.
An output specifying step in which the zero output specifying means performs a calculation for identifying the neuron computer whose output becomes zero according to the input data.
A control step in which the control means controls the calculation of the neural net computer so as to omit the calculation of the neuron computer whose output becomes zero based on the calculation result of the zero output specifying means.
A neural calculation method characterized by including.

Images