WO2025105520A1 - Lightweight deep learning model quantization method - Google Patents

Abstract

A lightweight deep learning model quantization method is provided. A deep learning model quantization method according to an embodiment of the present invention comprises: storing and accumulating maximum values of deep learning operations during training of a deep learning model; calculating an average of the accumulated maximum values; updating a quantization scale with the calculated average; and quantizing the deep learning model on the basis of the updated quantization scale. Accordingly, a quantization scale in a next training step is predicted and updated on the basis of a parameter distribution in a previous training step, thereby enabling fast and high-performance training in quantization training of a hardware accelerator model in a mobile device which is a resource-constrained environment.

Description

A lightweight deep learning model quantization method

The present invention relates to a method for quantizing a deep learning model, and more specifically, to a method for updating a quantization scale for a hardware accelerator-based deep learning model installed in a mobile device.

In conventional computer vision, most quantization-based object classification and object detection learning models are software-based models. The core of the quantization technology, real-time parameter value comparison for scale adjustment, is not easy to directly apply to a hardware accelerator-based design environment because it requires an additional process of re-verifying the output parameters for each layer.

In addition, in order to apply to mobile devices, a design that considers the small size of the learning model and the resulting storage memory space constraints is required, but design technology that satisfies this is lacking. Therefore, there is a problem that performance fluctuations occur depending on the model structure or the type of learning data due to the absence of adaptive quantization technology that satisfies this environment.

The present invention has been made to solve the above problems, and an object of the present invention is to provide a lightweight deep learning model quantization method that predicts and updates the quantization scale in the next learning step based on the parameter distribution in the previous learning step, as a quantization method applicable to a hardware accelerator model in a mobile device, which is a resource-limited environment.

According to one embodiment of the present invention for achieving the above purpose, a method for quantizing a deep learning model includes: a step of storing and accumulating maximum values of deep learning operations during training of a deep learning model; a step of calculating an average of the accumulated maximum values; a step of updating a quantization scale with the calculated average; and a step of quantizing a deep learning model based on the updated quantization scale.

The accumulation step may be to store the maximum values for each batch and accumulate them.

The calculation step could be to calculate the average of the maximum values stored per epoch.

The deep learning model quantization method according to the present invention may further include a step of performing learning of the next epoch on the quantized deep learning model.

Deep learning operations can be convolution operations.

The update step may update the quantization scale without knowing the distribution of the output feature map by the convolution operation.

Quantization can be symmetric quantization.

The accumulation step could be to store the maximum of the absolute values of the deep learning operation values.

Deep learning models can be deployed on mobile devices.

According to another aspect of the present invention, a deep learning operation device is provided, characterized by including: an operation unit that stores and accumulates maximum values of deep learning operations during training of a deep learning model, calculates an average of the accumulated maximum values, updates a quantization scale with the calculated average, and quantizes a deep learning model based on the updated quantization scale; and a memory that provides storage space required for the operation unit.

According to another aspect of the present invention, a method for quantizing a deep learning model is provided, characterized by including the steps of: updating a quantization scale with an average of maximum values of deep learning operations during training of a deep learning model; quantizing the deep learning model based on the updated quantization scale; and training the quantized deep learning model.

According to another aspect of the present invention, a deep learning operation device is provided, characterized by including: an operation unit that updates a quantization scale by an average of maximum values of deep learning operations during training of a deep learning model, quantizes the deep learning model based on the updated quantization scale, and trains the quantized deep learning model; and a memory that provides storage space required for the operation unit.

As described above, according to embodiments of the present invention, by predicting and updating the quantization scale in the next learning step based on the parameter distribution in the previous learning step, fast and high-performance learning is possible in quantizing a hardware accelerator model in a mobile device, which is a resource-constrained environment.

Figure 1. Example of problems occurring in hardware structure when applying quantization learning model in software.

Fig. 2. Example of learning process based on quantization scale update technique

Fig. 3. Learning method based on quantization scale update

Figure 4-5. Comparison of maximum and average distributions when applying the current epoch learning scale based on the previous distribution values.

Figure 6-9. Example of scale update process by epoch and batch unit during convolution operation process.

Fig. 10. Mobile deep learning computing device

Hereinafter, the present invention will be described in more detail with reference to the drawings.

Quantization technology for real-time operation is being widely used in various fields of object recognition related to mobile devices through cameras. In particular, lightweighting is a major challenge in implementing learning models in environments with limited memory, such as NPU hardware accelerator design.

However, most quantization models implemented on hardware are implemented after learning has been completed and quantization has been completed. There are several problems in implementing a quantization learning model suitable for the hardware structure by directly applying the quantization learning method in software.

Figure 1 is a diagram illustrating the general process of quantization learning in software and the problems that arise when the method is applied as is to a hardware accelerator.

When quantizing the convolution operation part in a CNN model, the quantization scale of the input feature map before convolution and the quantization scale of the output feature map after convolution have different values due to changes in the distribution value due to the intermediate operation process. In the case of the scale of the output feature map, a real-time value analysis and comparison process is required to quantize the parameters based on the accurate value distribution.

However, when analyzing output feature map values in real time on hardware, it is necessary to check all the values output after the convolution operation, so the feature map values are stored in memory once more during the comparison process. This process unconditionally consumes an additional cycle, which can be a fatal problem in hardware accelerator design.

Therefore, in order to configure a learning model that can be implemented in a hardware accelerator, the scale value for the output feature map must be known in advance. Fig. 2 is a diagram illustrating a model update method based on a quantization scale calculation method applicable to an embodiment of the present invention. A symmetric quantization structure is applied as the quantization structure. In the case of an asymmetric quantization structure, negative and positive values can be separately checked to minimize value loss occurring during quantization, but a separate calculation is added for the intermediate value that serves as a reference, so it is not suitable for hardware design.

In order to minimize the overall performance degradation of the model due to quantization, when quantization and calculation are performed on the convolution process, which involves the most calculations, the maximum absolute value of the values output after convolution is stored in a buffer.

The maximum value of the absolute value of the convolution output is updated and maintained during learning within the batch size during learning, and the maximum value is stored as a cumulative sum in a separate buffer at the end of each batch learning. The value stored as a cumulative sum is calculated as an average value at the unit epoch point when the dataset learning is completed once (the cumulative sum is divided by the number of batches) and applied as the quantization scale value of the next epoch. The same is true for the filter value, which updates the quantization scale on an epoch basis.

FIG. 3 is a diagram illustrating a quantization scale update method according to one embodiment of the present invention.

As shown, during the training of the deep learning model, the maximum value of the absolute value of the convolution output value is selected and stored for each batch. Since this is performed for each batch, the maximum values for each batch are accumulated. When training is completed for one epoch, the accumulated maximum values are divided by the number of batches to calculate the average of the maximum values.

The quantization scale is updated with the next calculated maximum value average, and the weights and activation functions of the deep learning model are quantized based on the updated quantization scale, and learning of the next epoch is performed on the quantized deep learning model.

In this way, in an embodiment of the present invention, quantization scale update is possible without understanding the distribution of the output feature map by convolution.

FIG. 4 and FIG. 5 show the difference between the distribution of the actual value and the distribution of the quantized scale value when the quantized scale update is performed according to the method suggested in the embodiment of the present invention. It can be confirmed that the scale value based on the average of the accumulated maximum value for each batch (Mean: red below) has a value suitable for the parameter distribution in the current epoch learning even though the feature map distribution is not identified in real time. On the other hand, there is a problem that the scale updated with only the simple maximum value (Prev. Xpoch Abs Max: blue above)) may cause a problem of performance degradation during learning because it is updated based on an excessively large value.

In this way, the quantization scale update through the average of the maximum values for each batch is impossible to operate in real time like software operation, but it is possible to check the average value for multiple image data and at the same time roughly understand the distribution change according to the overall learning tendency. In addition, it has the advantage of being able to implement a hardware accelerator because the process of understanding the real-time distribution parameters is excluded.

Figure 6-9 shows a more detailed process of a quantization scale update method according to an embodiment of the present invention for a convolution operation process. In a limited memory such as a mobile device, the batch size that can be learned at one time, i.e., the number of image data, cannot be small.

This means that the number of batch learnings per unit epoch is large, and the scale update technique calculated based on the average of the maximum values for each batch can derive the average for relatively more sample values in the limited environment, so it can be a stable method with a low influence from the bias for high values in a specific batch.

FIG. 10 is a diagram illustrating a configuration of a mobile deep learning computing device according to another embodiment of the present invention. As illustrated, the mobile deep learning computing device according to the embodiment of the present invention is configured to include a communication interface (110), a deep learning computing device (120), and a memory (130).

The communication interface (110) communicates with an external host system to receive a dataset and parameters of a pre-trained deep learning model. The deep learning operator (120) quantizes and trains the loaded deep learning model according to the method presented in FIG. 3 described above. The memory (130) provides the storage space required for the deep learning operator (120) to perform the operation.

So far, we have described in detail preferred embodiments of lightweight deep learning model quantization methods.

In the above embodiment, a method for quantization learning of a hardware accelerator model in a mobile device, which is an environment with limited resources, is presented by predicting and updating the average of the absolute values of the maximum values of deep learning operations per batch in the previous epoch at a quantization scale.

This enables fast and high-performance learning through a universal quantization scaling technique applicable to limited environment-based mobile devices as a quantization model applicable to hardware accelerator models.

Meanwhile, it goes without saying that the technical idea of the present invention can be applied to a computer-readable recording medium storing a computer program that performs the functions of the device and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention can be implemented in the form of a computer-readable code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, the computer-readable recording medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, etc. In addition, the computer-readable code or program stored on the computer-readable recording medium can be transmitted through a network connected between computers.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art without departing from the gist of the present invention as claimed in the claims. Furthermore, such modifications should not be individually understood from the technical idea or prospect of the present invention.

Claims (12)

A step of storing and accumulating the maximum values of deep learning operations during training of a deep learning model; A step of calculating the average of the accumulated maximum values; A step of updating the quantization scale with the computed average; and A method for quantizing a deep learning model, characterized by including a step of quantizing a deep learning model based on an updated quantization scale. In claim 1, The cumulative stage is, A deep learning model quantization method characterized by storing and accumulating maximum values by batch. In claim 2, The calculation steps are: A deep learning model quantization method characterized by calculating the average of the maximum values stored for each epoch. In claim 3, A method for quantizing a deep learning model, characterized in that it further includes a step of performing learning of the next epoch for the quantized deep learning model. In claim 1, Deep learning operations are, A deep learning model quantization method characterized by a convolution operation. In claim 5, The update steps are: A deep learning model quantization method characterized by updating the quantization scale without understanding the distribution of the output feature map by the convolution operation. In claim 1, Quantization is, A deep learning model quantization method characterized by symmetric quantization. In claim 7, The cumulative stage is, A deep learning model quantization method characterized by storing the maximum value of the absolute value of deep learning operation values. In claim 1, The deep learning model is, A method for quantizing a deep learning model, characterized by being installed on a mobile device. An operator that stores and accumulates the maximum values of deep learning operations during training of a deep learning model, calculates the average of the accumulated maximum values, updates the quantization scale with the calculated average, and quantizes the deep learning model based on the updated quantization scale; and A deep learning computing device characterized by including a memory that provides storage space required for the computing device. A step of updating the quantization scale by the average of the maximum values of deep learning operations during training of a deep learning model; A step of quantizing a deep learning model based on the updated quantization scale; and A method for quantizing a deep learning model, characterized by comprising a step of training a quantized deep learning model. An operator that updates the quantization scale by the average of the maximum values of deep learning operations during training of a deep learning model, quantizes the deep learning model based on the updated quantization scale, and trains the quantized deep learning model; and A deep learning computing device characterized by including a memory that provides storage space required for the computing device.

Images