WO2024135861A1 - Deep learning training method applying variable data representation type and mobile device applying same - Google Patents

Abstract

Provided are a deep learning training method applying a variable data representation type and a mobile device applying same. The deep learning network training method according to an embodiment of the present invention comprises: configuring a data representation type for each of layers constituting a deep learning network; and training the deep learning network while changing a representation type of data, which is input into each layer, according to the configured data representation type. Accordingly, by applying a variable data representation type for each layer and each channel when performing additional training or retraining, it is possible to perform additional training and retraining for a deep learning model on a mobile device with limited memory resources and power without a significant reduction in accuracy.

Description

Deep learning learning method applying variable data phenotype and mobile device to which it is applied

The present invention relates to deep learning learning, and more specifically, to a method of additionally learning and re-learning a model that has been trained on a server on a mobile device.

In order to operate a model that has been trained on a server on a new device, re-learning must be performed to regenerate deep learning parameters using the data used for learning and the data used for testing.

In other words, it is being developed by re-learning the deep learning model of a new device to generate deep learning parameters with minimal performance loss and then updating them in the final application.

At this time, when learning is conducted through a server, FP64 to FP32 are mainly used, and when low-specification server-level hardware is used, learning is performed with FP16. However, this is a level that can be sufficiently learned on servers with abundant power and sufficient hardware resources, but problems arise such as power consumption and insufficient hardware resources to operate on mobile devices.

The present invention was created to solve the above problems, and the purpose of the present invention is to provide a method for additional learning or relearning in mobile devices with limited memory resources, and a deep learning computing device and method using variable data phenotypes. In providing.

In order to achieve the above object, a deep learning network learning method according to an embodiment of the present invention includes the steps of setting a data phenotype for each layer constituting a deep learning network; It includes the step of training a deep learning network while changing the phenotype of the data input to each layer according to the set data phenotype.

The setting step may be setting the number of bits in the exponent part and setting the number of bits in the mantissa part for each layer.

The setting step may be setting the data representation for each channel for each layer.

The deep learning network training method according to the present invention may further include quantizing a training dataset to be used for deep learning network training.

The learning step may be to modify the phenotype of the data between layers from the phenotype of the previous layer to the phenotype of the next layer.

The setting step may be setting the data expression for each layer based on the data expression received from the outside.

Data phenotype can be determined based on the importance of each layer.

According to another aspect of the present invention, a control unit that sets data phenotypes for each layer constituting a deep learning network; and a data conversion unit that changes the phenotype of data input to each layer according to the data phenotype set by the control unit. A deep learning network device is provided, comprising a deep learning accelerator that trains a deep learning network with data whose phenotype is changed by a data conversion unit.

As described above, according to embodiments of the present invention, by applying variable data phenotypes for each layer and channel when performing additional learning or relearning, there is a significant reduction in accuracy in mobile devices with limited memory resources and power. It is possible to perform additional learning and retraining of the deep learning model without it.

1 is a diagram showing the inference process (forward path) in the learning process;

Figure 2 is a diagram showing the backpropagation process (Backward path) in the learning process;

Figure 3 shows equations showing the weight update concept,

Figure 4 shows the data phenotype concept,

5 is a variable data phenotype,

Figure 6 shows learning performance using data suggested phenotypes,

Figure 7 is a mobile device according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

Currently, most deep learning networks are trained on servers with multiple high-performance GPUs and large amounts of memory, so there are no problems caused by computing resources.

However, in cases where resources and power are limited, such as mobile devices, hardware development must proceed in a way that reduces the amount of calculation required and memory usage for intermediate data calculation.

In particular, additional learning or re-learning using large amounts of original learning data and FP32 or higher computational phenotypes is not possible on ASIC and FPGA platforms dedicated to learning for mobile devices, so technology to supplement this is required.

Since artificial intelligence learning requires enormous use of communication data with external memory, there are many applications that require on-device learning technology optimized for mobile devices.

Learning largely consists of an inference process and a back-propagation process. The forward path, which is the inference process, is shown in Figure 1, and the backward path, which is the backpropagation process, is shown in Figure 2.

In order to learn a deep learning network, after going through an inference process based on the current weight, the error value is calculated, the gradient is calculated, and the previous weight is updated. This is expressed in a formula as shown in Figure 3. At the server end, calculations for all formulas in FIG. 3 are performed using data representations of FP32 or higher.

However, in resource-limited mobile devices, like servers, all data sets are stored at the FP32 level, so hardware memory space is limited, and only limited bit-size and parallelism are possible due to the increase in hardware area of the calculator.

To solve this, FP16, which usually uses fewer bits, is used, or hardware is configured using fixed points to reduce the size of the operator. However, this also has the disadvantage of slowing the learning speed due to bandwidth problems occurring when inputting and outputting data to and from external memory, and lowering the learning accuracy due to the low accuracy of the calculator.

Accordingly, an embodiment of the present invention presents a deep learning learning method applying variable data phenotypes. This is a technology that processes data (storage, calculation, transmission, etc.) by changing it into a flexible data format according to a specific distribution so that various phenotypes (fixed point, floating point, etc.) shown in Figure 4 can be applied for learning data processing and calculation.

First, before changing the learning data of the deep learning network into the desired phenotype, by applying scale/exponent/bias to quantize the data to 8 bits or less and convert the data into a quantized value, it can be reduced to 1/8 of the existing data phenotype. . Quantized data can be restored to data values similar to the existing data phenotype using De-Quantization.

Next, set the data phenotype for each layer that makes up the deep learning network. In other words, the data expression can be set differently for each layer. By distinguishing between important and non-important layers, the data expression can be set differently.

Additionally, within the layer, data expression is set for each channel. In other words, the data phenotype can be set differently for each channel. By distinguishing between important channels and unimportant channels, the data phenotype can be set differently.

Figure 5 shows variable data phenotypes that can be set. In Figure 5, PEE (Partial Exponent Expression) represents the exponent part and Revised FP represents the mantissa part, and the data expression can be set by flexibly changing the number of bits in both the exponent part and the mantissa part. The data phenotype can be set/controlled by receiving the data phenotype set from an external source, such as the host.

Meanwhile, the size of Revised FP can be changed to a variety of bits depending on hardware resources. In the case of Revised FP, if the operation is performed with an existing fixed point, all bits are considered as mantissa, and when processing with FP operation, small exponent expression and mantissa are used. It can be expressed by dividing it into .

In the learning process, the deep learning network is trained by changing the phenotype of the data input to each layer according to the set data phenotype, that is, changing the phenotype of the data between layers from the phenotype of the previous layer to the phenotype of the next layer.

Meanwhile, in the learning process, the weight and gradient distributions are stochastically biased, so the meaningful dynamic range is not wide, so learning is possible with only small exponent phenotypes. This can be confirmed in the result table presented in Figure 6.

Since the method according to the embodiment of the present invention is mainly used in the low-bit or data pre-processing process during hardware processing, it can operate similarly to existing data input by adding simple hardware logic, resulting in performance similar to original-level learning. It is possible to derive

As a result of measuring this using public data sets (CIFAR-10, ImageNet, etc.), when using the data phenotype presented in the embodiment of the present invention, the measured performance showed a performance deterioration compared to FP32 in most phenotypes, as shown in Figure 6. It can be seen that there is almost no .

In addition, in the case of layer/channel data phenotypes that do not require the user, memory usage can be further reduced by using the minimum number of bits in the learning process, and the intermediate data storage space for the learning data can be used as additional batch data. It is also possible to increase the size.

Figure 7 is a block diagram of a mobile device according to an embodiment of the present invention. A mobile device according to an embodiment of the present invention is configured to include a memory 110, an MCU 120, a deep learning accelerator 130, a quantization unit 140, and a data conversion unit 150, as shown. .

The MCU 120 receives some learning data from the server, stores it in the memory 110, and sets the data phenotype for each layer/channel of the deep learning network. The quantization unit 140 performs quantization on the training data stored in the memory 110, and the data conversion unit 150 converts the quantized training data into a phenotype set in each layer/channel.

The deep learning accelerator 130 further trains or retrains the deep learning model learned on the server using learning data that is quantized by the quantization unit 140 and then converted into phenotype by the data conversion unit 150.

So far, the deep learning learning method using variable data phenotypes and the mobile device to which it is applied have been described in detail with preferred embodiments.

In an embodiment of the present invention, in order to reduce hardware resources used in the learning process and reduce power when relearning is performed using only specific data rather than relearning using existing learned original data, variable data phenotypes are used for each layer/channel. It was made applicable.

As a result, deep learning network learning was made possible at high speed with low energy consumption with only low accuracy loss when learning deep learning networks using mobile devices.

Meanwhile, of course, the technical idea of the present invention can be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to this embodiment. Additionally, the technical ideas according to various embodiments of the present invention may be implemented in the form of computer-readable code recorded on a computer-readable recording medium. A computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, of course, computer-readable recording media can be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, etc. Additionally, computer-readable codes or programs stored on a computer-readable recording medium may be transmitted through a network connected between computers.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

Claims (8)

Setting a data phenotype for each layer constituting a deep learning network; A deep learning network learning method comprising: training a deep learning network while changing the phenotype of data input to each layer according to the set data phenotype. In claim 1, The setup steps are: A deep learning network learning method characterized by setting the number of bits in the exponent part and setting the number of bits in the mantissa part for each layer. In claim 1, The setup steps are: A deep learning network learning method characterized by setting data phenotypes for each channel for each layer. In claim 1, A deep learning network learning method further comprising the step of quantizing a learning dataset to be used for deep learning network learning. In claim 1, The learning stage is, A deep learning network learning method characterized by modifying the phenotype of data between layers from the phenotype of the previous layer to the phenotype of the next layer. In claim 1, The setup steps are: A deep learning network learning method characterized by setting the data phenotype for each layer based on the data phenotype received from the outside. In claim 1, The data phenotype is, A deep learning network learning method characterized in that it is determined based on the importance of each layer. A control unit that sets the data phenotype for each layer constituting the deep learning network; and a data conversion unit that changes the phenotype of data input to each layer according to the data phenotype set by the control unit; A deep learning network device comprising a deep learning accelerator that trains a deep learning network with data whose phenotype is changed by a data conversion unit.

Images