WO2025089498A1 - Low-precision floating-point friendly quantization method and apparatus - Google Patents

Abstract

A low-precision floating-point friendly quantization method and apparatus are provided. A quantization apparatus of the present invention outputs a reduced-precision floating-point deep learning model from an input model. The quantization apparatus according to a preferred embodiment of the present invention comprises: a target layer detection unit for detecting a target layer in an input model; a channel detection unit for detecting a problematic channel for the layer detected by the target layer detection unit; and a quantization unit for selecting weights to which a predetermined quantization method is to be applied for the channel detected by the channel detection unit, and applying the predetermined quantization method to the selected weights.

Description

Low-precision floating-point friendly quantization method and device

The present invention relates to a low-precision floating-point friendly quantization method and device.

In a CNN (Convolutional Neural Network), multiple convolutional layers and multiple fully connected layers can be provided between the input layer and the output layer. When processing an image using a CNN, one convolutional layer can have as many filters as the number of channels of the input image, and the output image of the convolutional layer is generated by applying the filter assigned to each channel. For example, in a convolutional layer that applies a 3x3 sized filter to a 4x4x1 tensor-shaped input image, a 2x2x1 tensor-shaped image is generated through a convolution operation on the image and the filter. FP32 is generally used as a format for expressing the weights that constitute the filter, but it requires a lot of computational power and memory resources.

Therefore, in order to efficiently execute neural networks, quantization is often performed. The weights and activations of the network can be quantized. Quantization can be performed to convert them into a data type with a lower bit rate than the most commonly used basic format, FP32.

The types being converted include fixed-point formats such as INT16, INT8, and INT4, and floating-point formats such as FP16 and FP8. Because they use a lower bit width than the default FP32, they are also expressed as reduced precision.

FP32 and FP16 are standardized in IEEE-754 format, but the remaining formats, such as INT8 and FP8, are not standardized. Therefore, there are various detailed implementations depending on the developer. In addition, there are various formats for neural networks, such as BFLOAT16, Flexpoint, and Tensor Float 32.

Running a model with reduced precision has the advantage of reducing execution time due to reduced computational effort and memory usage. However, it has the disadvantage of reducing accuracy due to information loss caused by using a lower bit width to represent numbers.

There are two main ways to create a model by applying quantization. There is the Quantization-Aware Training (QAT) method, which learns by reflecting the effect of reduced precision during model training, and the Post-Training Quantization (PTQ) method, which converts a model pre-trained in FP32 to reduced precision.

The QAT method can create a model suitable for the target type and produce high accuracy results because the effect of reduced precision is reflected during model learning, but training data is required for learning.

The PTQ method does not require training data, but it has a problem in that the accuracy performance is low due to the loss that occurs when applying the reduced precision type. Calibration can be performed to solve this problem.

Calibration is the process of running a model in advance using sample data (calibration set) and checking the statistical distribution of values within the model. Through this process, the values of some of the variables used in the quantization process can be appropriately set.

Floating-point types are divided into sign, exponent, and mantissa parts. For example, the IEEE-754 single-precision floating-point type has 1 sign bit, 8 exponent bits, and 23 mantissa bits. Types other than FP32 have different bit configurations for the exponent and mantissa. For example, FP16 has 1 sign bit, 5 exponent bits, and 10 mantissa bits.

The FP8 type recently used in deep learning has no fixed standard. The configuration of sign-exponent-mantissa is mainly attempted in various ways such as 1-5-2, 1-4-3, 1-3-4, 1-2-5, etc., and the details may be slightly different depending on the developer. However, the basic form of the formula is the same as the IEEE-754 single-precision floating-point type.

That is, the dynamic range is controlled by the exponent, and the precision is controlled by the mantissa. In the case of the INT8 type, 7 bits are used for precision if the sign is expressed, and 8 bits are used if the sign is not expressed, while FP8 uses fewer bits for precision. FP quantization increases the quantization error as the number increases.

Among the operations of neural networks, there is a depthwise convolution operation. This operation is a core element of well-known networks such as MobileNet and EfficientNet, and is also widely used in other networks. Depthwise convolution is a convolution operation that performs operations independent of the input channel. Unlike general convolution, this operation does not accumulate values for the input channels. In other words, it performs operations on elements as large as the filter size of the depthwise convolution. For example, if it is a 2-dimensional depthwise convolution operation (Pytorch: DWConv2D (k = 3)) with a filter size of 3, it performs matrix multiply and accumulation operations on the 3 * 3 region of the input tensor and the 3 * 3 region of the filter.

However, depth-wise convolution operations tend to have a large decrease in accuracy when applying reduced-precision floating-point operations. Even in the case of general convolution, if the input channels are small and the number of elements to be operated is small, errors may occur significantly when using reduced-precision floating-point types (e.g. FP8), resulting in a decrease in accuracy.

The present invention has been made in consideration of these points, and aims to provide a quantization method capable of improving the decrease in accuracy that occurs when general quantization is applied.

Another object of the present invention is to provide a quantization method which can significantly reduce the amount of computation, memory usage and power consumption when using reduced precision.

The quantization device of the present invention outputs a reduced-precision floating-point deep learning model from an input model. The quantization device according to a preferred embodiment of the present invention comprises a target layer detection unit that detects a target layer from an input model, a channel detection unit that detects a problematic channel for the layer detected by the target layer detection unit, and a quantization unit that selects weights to which a predetermined quantization method is to be applied for the channel detected by the channel detection unit, and applies the predetermined quantization method to the selected weights.

In one embodiment, the target layer detection unit detects a depth-based convolution or a convolution having a predetermined number or less of input channels as a target layer. In one embodiment, the channel detection unit calculates a predetermined metric for each channel of the target layer, and determines that the channel is problematic if the metric satisfies a predetermined condition. As the predetermined condition, one of the following may be used: the calculated metric being equal to or greater than a predetermined threshold value; or the calculated metric being equal to or greater than a certain upper percentage when sorted by size. The metric is defined to indicate how widely the weights are distributed or the sizes of the weights. In one embodiment, the metric may use any one of the mean square error of weight quantization error, the sum of weight quantization errors, the variance of weights per channel, the maximum value of weights per channel, and the square of the maximum value of weights per channel.

In one embodiment, the quantization unit may select weights to which the predetermined quantization method is to be applied by using any one of a method of selecting a predetermined number of weights having the largest absolute value of quantization error in the corresponding channel, a method of selecting a weight having the largest exponent value, and a method of selecting all of the weights of the corresponding channel.

In one embodiment, the predetermined quantization method may be any one of a first quantization method (Rounding Away From Zero) that selects a value far from 0, a second quantization method (Rounding Towards Zero) that selects a value close to 0, and a method of selecting a combination in which the sum of squares of quantization errors for the entire target weight is minimized when both the first quantization method and the second quantization method are applied.

A quantization method according to a preferred embodiment of the present invention is a quantization method for outputting a reduced-precision floating-point deep learning model from an input model, comprising: a target layer detection step for detecting a target layer in an input model; a channel detection step for detecting a problematic channel for the layer detected by the target layer detection unit; and a quantization step for selecting weights to which a predetermined quantization method is to be applied for the channel detected by the channel detection unit, and applying the predetermined quantization method to the selected weights.

In one embodiment, in the target layer detection step, a depth-based convolution or a convolution having a predetermined number or less of input channels is detected as the target layer.

In one embodiment, the channel detection step calculates a predetermined metric for each channel of the target layer, and determines that the channel is problematic if the metric satisfies a predetermined condition. The predetermined condition may be one of the following: that the calculated metric is equal to or greater than a predetermined threshold value, or that the calculated metric is equal to or greater than a certain upper percentage when sorted by size. As the metric, any one of the mean square error of weight quantization error, the sum of weight quantization errors, the variance of weights per channel, the maximum value of weights per channel, and the square of the maximum value of weights per channel may be used.

In one embodiment, the quantization step may select weights to which the predetermined quantization method is to be applied by using any one of a method of selecting a predetermined number of weights having the largest absolute value of quantization error in the corresponding channel, a method of selecting a weight having the largest exponent value, and a method of selecting all of the weights of the corresponding channel.

In one embodiment, the predetermined quantization method may be any one of a first quantization method (Rounding Away From Zero) that selects a value far from 0, a second quantization method (Rounding Towards Zero) that selects a value close to 0, and a method of selecting a combination in which the sum of squares of quantization errors for the entire target weight is minimized when both the first quantization method and the second quantization method are applied.

According to one embodiment of the present invention, by finding a channel that is problematic in a depth-based convolution operation and then using a quantization method according to the present invention, it is possible to significantly recover the accuracy degradation that occurs when applying general quantization.

According to one embodiment of the present invention, when using reduced precision, computational amount, memory usage and power consumption can be reduced.

According to one embodiment of the present invention, in the case of general convolution, even if the input channel is small and the number of elements to be operated is small, the accuracy degradation can be recovered.

FIG. 1 is a block diagram showing the configuration of a low-precision floating-point friendly quantization device according to a preferred embodiment of the present invention.

FIG. 2 is a flowchart showing the operation flow of a low-precision floating-point friendly quantization method according to a preferred embodiment of the present invention.

Figure 3 is a diagram explaining an example of selecting weights according to index values.

Figure 4 is a diagram illustrating several examples of quantization methods.

Hereinafter, some embodiments of the present disclosure will be described in detail using exemplary drawings. When adding reference numerals to components of each drawing, it should be noted that the same numerals are used for identical components as much as possible even if they are shown in different drawings. In addition, when describing the present disclosure, if it is determined that a specific description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description thereof will be omitted.

In describing components of embodiments according to the present disclosure, symbols such as first, second, i), ii), a), b), etc. may be used. These symbols are only for distinguishing the components from other components, and the nature, order, or sequence of the corresponding components are not limited by the symbols. When a part in the specification is said to 'include' or 'provide' a component, this does not mean that other components are excluded, but rather that other components can be further included, unless explicitly stated to the contrary. In addition, terms such as 'part' and 'module' described in the specification mean a unit that processes at least one function or operation, and this can be implemented by hardware, software, or a combination of hardware and software.

The following description of the invention, together with the accompanying drawings, is intended to explain exemplary embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced.

The following description is based on FP8 for convenience, but reduced precision floating point can have various bit widths, such as FP8, FP10, FP6, and FP4. FP16 is considered to have a precision close to FP32 and is not considered reduced precision. However, all floating point data types less than 16 bits can be considered reduced precision.

Figure 1 is a block diagram showing the configuration of a low-precision floating-point friendly quantization device according to a preferred embodiment of the present invention. A deep learning model developed with high precision can be input to the floating-point friendly quantization device. For example, an FP16 or FP32 deep learning model can be input.

A floating-point friendly quantization device according to one embodiment of the present invention comprises a target layer detection unit (110), a channel detection unit (120), and a quantization unit (130).

The target layer detection unit (110) detects a target operation or layer in the input model. The target layer detection unit (110) can detect a depth-based convolution or a convolution with a small input channel.

The channel detection unit (120) detects a problematic channel for the operation (layer) detected by the target layer detection unit (110). In one embodiment, the channel detection unit (120) calculates a predetermined metric for each channel of the target layer and finds a case where the metric is large. In order to detect a problematic channel, it is preferable to measure the metric for each channel. As the metric, any one of the mean square error of weight quantization error, the sum of weight quantization errors, the variance of weights for each channel, the maximum value of weights for each channel, and the square of the maximum value of weights for each channel can be used.

The quantization unit (130) applies the proposed quantization method to the channel detected by the channel detection unit (120). In one embodiment, the quantization unit (130) quantizes in a different way from the existing RTN (Rounding to nearest) method that quantizes to the nearest value. The quantization unit (130) selects weights to which the different quantization method is applied in order to reduce errors in a channel that generates many quantization errors, and applies the different quantization method to the selected weights. Various methods may be used as weight selection methods, such as a method of selecting a predetermined number of weights having the largest absolute value of quantization error in the corresponding channel, a method of selecting a weight having the largest exponent value, and a method of selecting all of the weights of the corresponding channel. Other quantization methods that can be used include Rounding Away From Zero (RAFZ) that selects values far from 0, Rounding Towards Zero (RTZ) that selects values close to 0, and a method that selects a combination that minimizes the sum of squares of quantization errors for all target weights when applying both RAFZ and RTZ.

Through this process, a reduced-precision floating-point deep learning model is output. That is, the same model that uses a bit-width smaller than the precision of the input model is output.

Figure 2 is a flow chart showing the operation flow of a low-precision floating-point friendly quantization method according to a preferred embodiment of the present invention. A deep learning model developed with high precision can be input to a floating-point friendly quantization device. For example, an FP16 or FP32 deep learning model can be input.

A floating-point friendly quantization device detects a target operation or layer from an input model (step S110). A problematic channel is detected for the detected operation (layer) (step S120). A proposed quantization method is applied to the detected channel (step S130). Through this process, a reduced-precision floating-point deep learning model is output. In other words, the same model using a bit-width smaller than the precision of the input model is output.

Each operation step is described in detail below.

1. Target motion detection (step S110)

Detects depth-based convolutions or convolutions with small input channels. The criteria for whether the input channels are small can be set by the user. Preferably, it can be determined that the number of input channels is small if it is 3 or less, but it is not limited to this value. It is also possible to target convolutions with larger input channels, in which case additional operations may be required. Although the performance (accuracy) after applying the method of the present invention does not deteriorate further, it is not guaranteed that the performance will be improved even if the method of the present invention is applied when the input channels are large.

2. Target channel detection method (step S120)

A predetermined metric is calculated for each channel of the target layer and the case where this metric is large is found. The predetermined metric should be a metric that can represent the wide distribution of the weights among the input tensor and the weights, which are the operation inputs, or the size of the values. In order to detect the problematic channel, the metric is measured for each channel.

For example, the following metrics can be used as metrics. In the equation below, W_FP32 represents a weight expressed in the FP32 type, and W_target_type represents a weight expressed in the target type, for example, the FP8 type.

In the following, the metrics are described based on two dimensions, and accordingly, two letters i and j are used. In other embodiments, the metrics can be applied to one dimension or three dimensions. In one-dimensional metrics, only the letter i is used, and in three-dimensional metrics, three letters, i, j, and k, are used.

< Metric 1 >

The mean squared error of the weight quantization error, expressed as mathematical expression 1, can be used as a metric.

< Metric 2 >

The sum of weighted quantization errors expressed as in mathematical expression 2 can be used as a metric.

< Metric 3 >

The variance of channel-specific weights, expressed as in mathematical expression 3, can be used as a metric.

< Metric 4 >

The maximum value of the channel-specific weights expressed as in mathematical expression 4 can be used as a metric.

< Metric 5 >

The square of the maximum value of the channel-specific weights, expressed as in mathematical expression 5, can be used as a metric.

A metric being "large" can be judged as being greater than some arbitrary threshold value obtained experimentally, or as being in the upper range when the metrics are sorted by size. A "top range" can be defined as being greater than a certain percentage, such as the top 10%.

The lower the threshold value or the lower the ratio value, the more target channels there are, and the higher the possibility of recovering accuracy. Depending on which solution method is applied to the target channel, the execution overhead (execution time of the result model, complexity of applying the new method, etc.) will vary, so it cannot be made large unconditionally. Also, even if it is made large unconditionally, the accuracy degradation will not be recovered indefinitely. Therefore, it is desirable to select an appropriate value.

3. Quantization method applied to the problem channel (step S130)

In order to reduce errors in channels that generate a lot of quantization errors, the following method can be applied. The target channel has ^k2 weights, and among these weights, weights to which a new quantization method is to be applied are selected, and the new quantization method is applied to the selected weights. Meanwhile, in n-dimensional convolution and deconvolution, the number of weights can be k ⁿ .

Among the k ² weights existing in the channel, the target weight can be specified in various ways as follows.

< Weight selection method 1 >

This is a method of selecting a predetermined number of weights with the largest absolute value of quantization error in the corresponding channel. The quantization error can be defined as the value obtained by subtracting the value when converted to the target type from the value of W_FP32. In one embodiment, if the two with the largest absolute values of quantization errors are selected, one positive weight and one negative weight can be selected.

< Weight selection method 2 >

The weight with the largest exponent is selected. That is, when the weights are converted to the target type, all weights with the largest exponent among sign-exponent-decimal are selected. A large exponent means that the size of the corresponding weight is large. In another embodiment, the largest and second largest exponents can be selected.

FIG. 3 illustrates an example of selecting weights according to exponent values. Method 2-1 is for selecting the largest exponent value, and Method 2-2 is for selecting the largest and second largest exponent values. In the example of FIG. 3, in the case of Method 2-1, weights 4 and 6, which have the largest exponent value of "1100", are selected, and in the case of Method 2-2, weights 4 and 6, which have the largest exponent value of "1100", and weights 7 and 9, which have the second largest exponent value of "1011", are selected.

< Weight selection method 3 >

You can select all the weights of a given channel, that is, you select all the weights, not just those that satisfy a specific condition.

In general, when quantizing, real numbers are rounded to the nearest integer value. For example, as shown in (a) of Fig. 4, -3.36, which is between -3.5 and -3.25, is quantized to -3.25, which has a smaller difference value, since the difference between -3.5 and -3.25 is 0.14 and the difference between -3.25 and -3.36 is 0.12. 3.19, which is between 3 and 3.25, is quantized to 3.25, which has a smaller difference value of 0.06 than 3.00, which has a difference value of 0.19.

In the present invention, when quantizing selected weights, one of the following quantization methods can be applied instead of the general rounding method.

< Quantization method 1 > Rounding away from zero

Rounding to values farther from 0. For example, as shown in (b) of Fig. 4, -3.36, which is a value between -3.5 and -3.25, is quantized to -3.5, which is a value farther from 0 than -3.25. 3.19, which is a value between 3 and 3.25, is quantized to 3.25, which is a value farther from 0 than 3.00.

< Quantization method 2 > Rounding towards zero

Rounding to a value closer to 0. For example, as shown in (c) of Fig. 4, -3.36, which is a value between -3.5 and -3.25, is quantized to -3.25, which is a value closer to 0 than -3.5. 3.19, which is a value between 3 and 3.25, is quantized to 3.00, which is a value closer to 0 than 3.25.

< Quantization method 3 >

When applying both quantization method 1 (Rounding away from zero) and quantization method 2 (Rounding towards zero) to the target weights, the combination that minimizes the sum of the squares of the quantization errors for the entire target weights is selected.

The accuracy was compared when the existing FP32 format was quantized to NF8 format and when the method of the present invention was applied to the entire depth-wise convolution. When applied to MobileNetV1, the existing method quantized to NF8 format recorded an accuracy of 81% compared to the FP32 format, whereas the present method quantized to NF8 format recorded an accuracy of 94% compared to the FP32 format. When applied to MobileNetV2, the existing method quantized to NF8 format recorded an accuracy of 91% compared to the FP32 format, whereas the present method quantized to NF8 format recorded an accuracy of 97% compared to the FP32 format.

Each component of the device or method according to the present invention may be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component may be implemented as software, and a microprocessor may be implemented to execute the function of the software corresponding to each component.

Various implementations of the systems and techniques described herein can be implemented as digital electronic circuits, integrated circuits, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementations of one or more computer programs executable on a programmable system. The programmable system includes at least one programmable processor (which may be a special purpose processor or a general purpose processor) coupled to receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device. Computer programs (also known as programs, software, software applications, or code) include instructions for the programmable processor and are stored on a "computer-readable medium."

A computer-readable recording medium includes any type of recording device that stores data that can be read by a computer system. Such a computer-readable recording medium can be a non-volatile or non-transitory medium, such as a ROM, a CD-ROM, a magnetic tape, a floppy disk, a memory card, a hard disk, a magneto-optical disk, a storage device, and may further include a transitory medium, such as a data transmission medium. In addition, the computer-readable recording medium can be distributed over a network-connected computer system, so that the computer-readable code can be stored and executed in a distributed manner.

Although the flowchart of this specification describes each process as being executed sequentially, this is only an illustrative description of the technical idea of one embodiment of the present disclosure. In other words, a person having ordinary skill in the art to which one embodiment of the present disclosure belongs may change the order described in the flowchart of this specification without departing from the essential characteristics of one embodiment of the present disclosure, or may modify and modify and apply various modifications and variations such as executing one or more of the processes in parallel. Therefore, the flowchart of this specification is not limited to a chronological order.

The above description is merely an example of the technical idea of the present embodiment, and those skilled in the art will appreciate that various modifications and variations may be made without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are not intended to limit the technical idea of the present embodiment, but rather to explain it, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The protection scope of the present embodiment should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present embodiment.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application claims the benefit of Korean patent application No. 10-2023-0145070, filed in Korea on October 26, 2023, Korean patent application No. 10-2023-0146559, filed in Korea on October 30, 2023, and Korean patent application No. 10-2023-0182301, filed in Korea on December 14, 2023, which are incorporated herein by reference in their entirety.

Claims (14)

A quantization device that outputs a reduced-precision floating-point deep learning model from an input model, A target layer detection unit that detects the target layer in the input model. A channel detection unit that detects problematic channels for the layers detected by the target layer detection unit, and A quantization unit that selects weights to which a predetermined quantization method is applied for the channel detected by the above channel detection unit, and applies the predetermined quantization method to the selected weights. A quantization device having a . In the first paragraph, The above target layer detection unit detects a depth-based convolution or a convolution with a predetermined number of input channels or less as a target layer. Quantization device. In the second paragraph, The above channel detection unit calculates a predetermined metric for each channel of the target layer and determines that the channel is problematic if the metric satisfies a predetermined condition. Quantization device. In the third paragraph, the predetermined condition is: One of the above calculated metrics being greater than or equal to a predetermined threshold value or one of the above calculated metrics being greater than or equal to a certain percentage of the top when sorted by size. Quantization device. In clause 3 or 4, The above metric is one of the mean square error of weight quantization error, sum of weight quantization error, variance of weight per channel, maximum value of weight per channel, and square of maximum value of weight per channel. Quantization device. In any one of claims 1 to 3, the quantization unit, Selecting weights to which the predetermined quantization method is to be applied by using any one of the following methods: a method of selecting a predetermined number of weights having the largest absolute value of quantization error in the corresponding channel, a method of selecting a weight having the largest exponent value, and a method of selecting all weights of the corresponding channel. Quantization device. In any one of claims 1 to 3, the predetermined quantization method is: One of the first quantization method (Rounding Away From Zero) that selects a value far from 0, the second quantization method (Rounding Towards Zero) that selects a value close to 0, and the method of selecting a combination that minimizes the sum of squares of quantization errors for the entire target weight when applying both the first quantization method and the second quantization method. Quantization device. A quantization method for outputting a reduced-precision floating-point deep learning model from an input model, Target layer detection step that detects the target layer in the input model. A channel detection step for detecting problematic channels for the layer detected by the target layer detection unit, and A quantization step for selecting weights to which a predetermined quantization method is applied for the channel detected by the above channel detection unit, and applying the predetermined quantization method to the selected weights. A quantization method having a . In Article 8, The above target layer detection step detects a depth-based convolution or a convolution with a predetermined number of input channels or less as a target layer. Quantization method. In Article 9, The above channel detection step calculates a certain metric for each channel of the target layer and determines that the channel is problematic if the metric satisfies a certain condition. Quantization method. In the 10th paragraph, the predetermined condition is: One of the above calculated metrics being greater than or equal to a predetermined threshold value or one of the above calculated metrics being greater than or equal to a certain percentage of the top when sorted by size. Quantization method. In clause 10 or 11, The above metric is one of the mean square error of weight quantization error, sum of weight quantization error, variance of weight per channel, maximum value of weight per channel, and square of maximum value of weight per channel. Quantization method. In any one of claims 8 to 10, the quantization step comprises: Selecting weights to which the predetermined quantization method is to be applied by using any one of the following methods: a method of selecting a predetermined number of weights having the largest absolute value of quantization error in the corresponding channel, a method of selecting a weight having the largest exponent value, and a method of selecting all weights of the corresponding channel. Quantization method. In any one of claims 8 to 10, the predetermined quantization method is: One of the first quantization method (Rounding Away From Zero) that selects a value far from 0, the second quantization method (Rounding Towards Zero) that selects a value close to 0, and the method of selecting a combination that minimizes the sum of squares of quantization errors for the entire target weight when applying both the first quantization method and the second quantization method. Quantization method.

Images