WO2021143686A1 - Neural network fixed point methods and apparatuses, electronic device, and readable storage medium - Google Patents

Abstract

The present application provides a neural network fixed point method and apparatus, an electronic device, and a readable storage medium. The neural network fixed point method comprises: for a unit to be fixed-pointed of a neural network, on the basis of data distribution of feature values output by the unit to be fixed-pointed in the case that a fixed-pointed unit of the neural network maintains a fixed-pointed state, determining fixed-point hyperparameters of the feature values output by the unit to be fixed-pointed; and on the basis of the fixed-point hyperparameters, performing fixed point on the feature values output by the unit to be fixed-pointed. The method can improve the fixed-point performance of the neural network.

Description

Neural network fixed point method, device, electronic equipment and readable storage medium Technical field

This application relates to methods, devices, electronic equipment, and readable storage media for neural network targeting.

Background technique

Neural network fixed-point (also known as quantization) is a commonly used model acceleration algorithm, which quantizes the weight of the model and the output eigenvalue (the weight and the output eigenvalue are float type values) into fixed bits Numbers, such as 8bit, 16bit or very low 1, 2bit, can well solve the problems of complex multiplication and accumulation calculations and huge bandwidth caused by floating-point data on the hardware. The lower the number of quantized bits of the model, the more obvious the acceleration of the model on hardware.

In a neural network fixed-point solution, a certain number of sample pictures can be input to the original model at one time, the forward calculation of the model records the data distribution of the eigenvalues output by each layer, and then the eigenvalues output by each layer are calculated separately. The fixed-point super-parameters do not consider the impact of the fixed-pointing of the pre-layer on the post-layer, and the performance of the fixed-point neural network is low.

Summary of the invention

In view of this, this application provides a method, device, electronic device, and readable storage medium for neural network targeting.

Specifically, this application is implemented through the following technical solutions:

According to the first aspect of the embodiments of the present application, there is provided a neural network fixed-pointing method, including: for the pending unit of the neural network, based on the situation that the fixed-pointing unit of the neural network maintains the fixed-pointing state, the pending-pointing method is provided The data distribution of the characteristic value output by the unit determines the fixed-point hyperparameter of the characteristic value output by the to-be-fixed unit; based on the fixed-point super-parameter, the characteristic value output by the to-be-fixed unit is fixed.

According to a second aspect of the embodiments of the present application, there is provided a neural network fixed-pointing method, including: for a neural network to-be-fixed unit, based on the situation that the fixed-pointing unit of the neural network maintains a fixed-point state, the pending point Optimizing the weights of the unit to be fixed with the characteristic value output by the quantization unit and the eigenvalue output by the unit to be fixed in the original floating-point state; and fix the optimized unit to be fixed.

According to a third aspect of the embodiments of the present application, a neural network fixed-point method is provided, including: analyzing the topological structure of the input floating-point model to generate a neural network data flow graph; based on the sample picture information in the configuration file, Generate the data layer of the neural network; split and merge the topology of the neural network based on the optimization strategy of the platform to be deployed to obtain the preprocessed floating-point model; perform the preprocessing based on the above method The floating-point model is fixed-point.

According to a fourth aspect of the embodiments of the present application, there is provided a neural network fixed-pointing device, including: a determining module, used for a pending unit of the neural network, based on the situation that the fixed-pointed unit of the neural network maintains a fixed-pointing state Download the data distribution of the characteristic value output by the to-be-fixed unit to determine the fixed-pointing hyperparameter of the characteristic value output by the to-be-fixed unit; the fixed-pointing module is used for the output of the to-be-fixed unit based on the fixed-pointing hyperparameter The characteristic value is fixed-point.

According to a fifth aspect of the embodiments of the present application, there is provided a neural network fixed-pointing device, including: an optimization module, which is used to maintain a fixed-pointing state for the fixed-pointing unit of the neural network based on the fixed-pointing unit of the neural network In this case, the eigenvalue output by the to-be-fixed unit and the eigenvalue output by the to-be-fixed unit in the original floating-point state optimize the weight of the to-be-fixed unit; the fixed-point module is used to optimize the The to-be-fixed unit is fixed-pointed.

According to a sixth aspect of the embodiments of the present application, there is provided a neural network fixed-pointing device, including: an analysis module for analyzing the topological structure of an input floating-point model to generate a neural network data flow graph; a generation module, using Based on the sample picture information in the configuration file, the data layer of the neural network is generated; the processing module is used to split and merge the topology of the neural network based on the optimization strategy of the platform to be deployed to obtain preprocessing The latter floating-point model; a fixed-point module for fixed-pointing the pre-processed floating-point model based on the above method.

According to a seventh aspect of the embodiments of the present application, there is provided an electronic device including a processor and a machine-readable storage medium, the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and The processor is used to execute machine-executable instructions to implement the above-mentioned neural network fixed-point method.

According to an eighth aspect of the embodiments of the present application, there is provided a machine-readable storage medium that stores machine-executable instructions in the machine-readable storage medium, and when the machine-executable instructions are executed by a processor, the above-mentioned neural network fixed point is realized化 method.

According to a ninth aspect of the embodiments of the present application, a computer program is provided, the computer program is stored in a machine-readable storage medium, and when the computer program is executed by a processor, the above neural network fixed-pointing method is realized.

The technical solution provided in this application can at least bring about the following beneficial effects: For the to-be-fixed unit of the neural network, the data distribution of the characteristic value output by the to-be-fixed unit based on the situation that the fixed-point unit of the neural network maintains the fixed-point state , Determine the fixed-point hyperparameter of the eigenvalue output by the to-be-fixed unit, and based on the fixed-point super-parameter, fix the eigenvalue output by the to-be-fixed unit, and perform unit-by-unit fixed-pointing on the neural network. The to-be-fixed unit of the neural network keeps the fixed-point unit in a fixed-point state to obtain a more realistic data distribution of the output characteristic value, thereby improving the performance of the neural network's fixed-pointing.

Description of the drawings

Fig. 1 is a schematic flowchart of a neural network fixed point method shown in an exemplary embodiment of the present application.

Fig. 2 is a schematic flowchart of another neural network fixed-pointing method shown in an exemplary embodiment of the present application.

Fig. 3 is a schematic flowchart of a final fixed-pointing hyperparameter for determining the characteristic value output by the to-be-fixed unit from the first fixed-pointing superparameter candidate value range according to an exemplary embodiment of the present application.

Fig. 4 is a schematic flowchart of another neural network fixed point method according to an exemplary embodiment of the present application.

FIG. 5 is a schematic flowchart of a final fixed-pointing hyperparameter for determining the weight of the to-be-fixed unit from the second fixed-pointing super-parameter candidate value range according to an exemplary embodiment of the present application.

Fig. 6 is a schematic flowchart of a neural network fixed point method according to an exemplary embodiment of the present application.

Fig. 7 is a schematic flowchart of a neural network fixed-pointing method shown in an exemplary embodiment of the present application.

Fig. 8 is a schematic diagram of an automated neural network pointing system shown in an exemplary embodiment of the present application.

Fig. 9 is a schematic diagram of a functional flow chart of a fixed-point evaluation and tuning module shown in an exemplary embodiment of the present application.

Fig. 10 is a schematic diagram showing an implementation of an optimization strategy for minimizing local errors according to an exemplary embodiment of the present application.

Fig. 11 is a schematic structural diagram of a neural network pointing device according to an exemplary embodiment of the present application.

Fig. 12 is a schematic structural diagram of a neural network pointing device according to an exemplary embodiment of the present application.

Fig. 13 is a schematic structural diagram of a neural network pointing device according to an exemplary embodiment of the present application.

Fig. 14 is a schematic diagram showing the hardware structure of an electronic device according to an exemplary embodiment of the present application.

Detailed ways

The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the application as detailed in the appended claims.

The terms used in this application are only for the purpose of describing specific embodiments, and are not intended to limit the application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of this application, and to make the above-mentioned objectives, features, and advantages of the embodiments of this application more obvious and understandable, the technical solutions in the embodiments of this application are described below with reference to the accompanying drawings. Further detailed description.

Please refer to FIG. 1, which is a schematic flowchart of a neural network fixed-pointing method provided by an embodiment of this application. As shown in FIG. 1, the neural network fixed-pointing method may include the following steps S100 and S110.

Step S100: For the to-be-fixed unit of the neural network, based on the data distribution of the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, determine the characteristic value output by the to-be-fixed unit The fixed-point super-parameters.

In the embodiment of this application, considering that a certain number of sample pictures are input to the original model at one time to realize the fixed point of the output feature value of each layer, after the previous layer is fixed, the latter layer , The input is still the output in the floating-point state of the previous layer, and the authenticity of the data distribution is poor.

In order to obtain a more realistic data distribution of the eigenvalues output by each layer of the neural network, and to improve the performance of the neural network fixed point, you can follow the order of each layer of the neural network from front to back, with one layer or continuous multiple layers as a unit, Divide the neural network into multiple units, and fix the neural network unit by unit. When a unit of the neural network is fixed, keep the unit (which can be called the pending unit) before the unit (which can be called Is a fixed-point unit) is in a fixed-point state, and based on the data distribution of the feature value output by the to-be-fixed unit under the condition that the fixed-point unit remains in the fixed-point state, the fixed-point of the feature value output by the to-be-fixed unit is determined Super parameters.

In the embodiment of the present application, when the unit to be fixed is the first unit to be fixed in the neural network, for example, the first unit of the neural network, the neural network does not include the fixed unit, in this case, it can be based on the The data distribution of the characteristic value output by the to-be-fixed unit determines the fixed-point hyperparameter of the characteristic value output by the to-be-fixed unit, and fixes the characteristic value output by the to-be-fixed unit based on the fixed super parameter.

For example, suppose the neural network includes two units (assumed as unit 1 and unit 2), then unit 1 and unit 2 can be fixed in sequence. When the unit 1 is fixed (that is, the unit to be fixed is unit 1), a certain number of samples can be input to the neural network to calculate the data distribution of the eigenvalues output by the unit 1, and based on the eigenvalues output by the unit 1. The data distribution is to determine the fixed-point super-parameters of the eigenvalues output by the unit 1, and to fix the eigenvalues output by the unit 1 based on the determined fixed-point super-parameters.

After the fixed-pointing of unit 1 is completed, when unit 2 is fixed-pointed (that is, the unit to be fixed is unit 2), unit 1 can be kept in a fixed-point state, a certain number of samples are input to the neural network, and unit 2 is counted. Based on the data distribution of the output feature value, and based on the data distribution of the feature value output by the unit 2, the fixed-point hyperparameter of the feature value output by the unit 2 is determined.

In the embodiment of the present application, when the to-be-fixed unit of the neural network is fixed, if there are multiple fixed-point units before the to-be-fixed unit, it can also be used in some of the multiple fixed-point units. In the case that the fixed-point unit maintains the fixed-point state, the pending unit is fixed-pointed, and its specific implementation is not repeated here.

In step S110, the characteristic value output by the unit to be fixed is fixed based on the fixed super parameter.

In the embodiment of the present application, when the fixed-pointing hyperparameter of the characteristic value output by the to-be-fixed unit is determined in the manner described in step S100, the characteristic value output by the to-be-fixed unit can be fixed based on the fixed-pointing super-parameter. change.

It can be seen that in the method flow shown in Figure 1, the neural network is fixed unit by unit, and for the pending unit of the neural network, the fixed unit is kept in a fixed state to obtain a more realistic output feature value. Data distribution, thus, improve the performance of neural network fixed-point.

In a possible embodiment, referring to FIG. 2, in step S100, after determining the fixed-pointing hyperparameter of the feature value output by the to-be-fixed unit, the neural network fixed-pointing method provided in the embodiment of the present application may further include The following steps: step S101, determine a first fixed-point super-parameter candidate value range based on the fixed-point super-parameter of the characteristic value output by the to-be-fixed unit; step S102, from the first fixed-point super-parameter candidate value range Determine the final fixed-point hyperparameter of the eigenvalue output by the to-be-fixed unit.

In step S110, the characteristic value output by the to-be-fixed unit is fixed based on the to-be-fixed super-parameter, including: step S111, the final fixed-pointing super-parameter based on the characteristic value output by the to-be-fixed unit is The eigenvalues output by the pointization unit are fixed-pointed.

Exemplarily, when the fixed-point super-parameter of the characteristic value output by the to-be-fixed unit is determined in the manner described in step S100, the determined fixed-point super-parameter may also be optimized to obtain the actual use of the to-be-determined super-parameter. The eigenvalues output by the initiation unit are used for fixed-point fixed-point super-parameters (referred to as final fixed-point super-parameters in this article) to further improve the performance of the neural network fixed-point.

Exemplarily, based on the fixed-pointing hyperparameters of the feature values output by the to-be-fixed unit determined in step S100, the range for searching for the final fixed-pointing hyperparameters (herein referred to as the first fixed-pointing hyperparameter candidate value) may be determined. Range), and determine the final fixed-point super-parameter of the characteristic value output by the to-be-fixed unit from the first fixed-point super-parameter candidate value range.

Exemplarily, the first fixed-point hyperparameter candidate value range may be a numerical interval including the fixed-point hyperparameter determined in step S100.

For example, the lower limit and upper limit of the candidate value range of the first fixed-point super-parameter can be obtained by subtracting and adding a specific value greater than 0 on the basis of the fixed-point super-parameter determined in step S100, thereby determining the first fixed-point super-parameter. The range of hyperparameter candidate values is fixed. Alternatively, the fixed-point super-parameter determined in step S100 may be used as the lower limit of the first fixed-point super-parameter candidate value range, and a specific value greater than 0 can be added to the lower limit to obtain the first fixed-point super-parameter. The upper limit of the parameter candidate value range is determined, thereby determining the first fixed-point hyperparameter candidate value range.

The determination of the second fixed-pointed super-parameter candidate value range and the third fixed-pointed super-parameter candidate value range below can be obtained in the same way, and will not be repeated in the embodiments of the present application.

In an example, as shown in FIG. 3, step S102, determining the final fixed-pointing hyper-parameter of the characteristic value output by the to-be-fixed unit from the first fixed-pointing super-parameter candidate value range can be implemented by the following steps: Step S1021, respectively, determine the output error of the neural network corresponding to each candidate value in the first fixed-point hyperparameter candidate value range; step S1022, determine the candidate value with the smallest output error of the corresponding neural network as the pending point The final fixed-point hyperparameter of the eigenvalue output by the unit.

Exemplarily, based on the output error of the neural network, the fixed-point hyperparameter of the eigenvalue output by the fixed-point unit can be optimized, and the fixed-point hyperparameter that minimizes the output error of the neural network can be selected as the eigenvalue output by the to-be-fixed unit Finally, the super-parameters are fixed.

When the first fixed-point hyperparameter candidate value range is determined, the output error of the neural network corresponding to each candidate value in the first fixed-point hyperparameter candidate value range can be determined separately, and the output error of the corresponding neural network can be minimized The candidate value of is determined as the final fixed-pointing hyperparameter of the feature value output by the to-be-fixed-pointing unit.

In the embodiment of the present application, determining the candidate value with the smallest output error of the corresponding neural network as the final fixed-pointing hyperparameter of the characteristic value output by the to-be-fixed unit is only the determination of the fixed-pointing hyperparameter in the embodiment of this application. It is an implementation manner and is not a limitation of the protection scope of the present application. In the embodiments of the present application, the final fixed-pointing hyperparameter of the characteristic value output by the to-be-fixed unit can also be determined in other ways.

For example, the average value of the candidate values whose output error of the corresponding neural network is less than the preset threshold can be determined as the final fixed-pointing hyperparameter of the feature value output by the to-be-fixed unit. Alternatively, the candidate values can be sorted according to the output error of the corresponding neural network from small to large, and the average value of the previously preset number of candidate values after sorting can be determined as the final feature value output by the unit to be fixed. Fixed-point super-parameters.

In an example, in step S1021, respectively determining the output error of the neural network corresponding to each candidate value in the first fixed-point hyperparameter candidate value range may include: for any candidate value, based on the first type feature value and The second type feature value determines the output error of the neural network corresponding to the candidate value.

Exemplarily, considering the neural network in the original floating-point state, the fixed-point processing of the neural network may bring errors to the output accuracy of the neural network, and the output of the fixed-point neural network is different from the original floating-point state. The smaller the output error of the neural network, the higher the output accuracy of the fixed-point neural network. Therefore, the fixed-point can be fixed based on the error between the output of the fixed-point neural network and the output of the neural network in the original floating-point state Super parameters are optimized.

For any candidate value in the first fixed-point hyperparameter candidate value range, it is possible to determine the characteristic value output by each unit of the neural network when using the candidate value to fix the characteristic value output by the fixed-point unit (herein (Referred to as the first type feature value), and, in the original floating point state, the feature value output by each unit of the neural network (herein referred to as the second type feature value), determine the output error of the neural network corresponding to the candidate value, which The specific implementation may be described below in conjunction with specific examples, and the embodiments of the present application will not be repeated here.

In a possible embodiment, as shown in FIG. 4, the neural network positioning method provided in the embodiment of the present application may further include the following steps S400, S410, S420, and S430.

Step S400: For the to-be-fixed unit of the neural network, based on the data distribution of the weight of the to-be-fixed unit, determine the fixed-point hyperparameter of the weight of the to-be-fixed unit.

Step S410, based on the fixed-pointed super-parameter of the weight of the to-be-fixed unit, determine a second fixed-pointed super-parameter candidate value range.

Step S420: Determine the final fixed super parameter of the weight value of the to-be-fixed unit from the second fixed super parameter candidate value range.

In step S430, the weight of the unit to be fixed is fixed based on the final super-parameter of the weight of the unit to be fixed.

Exemplarily, in order to realize the fixed point of the neural network, for the to-be-fixed unit of the neural network, in addition to the characteristic value output by the to-be-fixed unit can be fixed according to the method described in the above embodiment, it also needs to be fixed. The weight of the unit is fixed-point.

Because in the traditional neural network fixed-point solution, for the neural network in the original floating-point state, the weight of the floating-point state of each unit has been determined when the neural network is trained (the neural network is in the original floating-point state at this time). The data distribution of the weights of the floating-point state of the unit is not affected by the fixation of the previous unit. Therefore, for the weights of the neural network, when the weights of the to-be-fixed units of the neural network are fixed, the The specific implementation can refer to the related implementation in the traditional neural network fixed-point solution.

Exemplarily, similar to the fixed-pointing of the feature value output by the to-be-fixed unit described in the foregoing embodiment, for the weight, when the fixed-pointing hyperparameter of the weight of the to-be-fixed unit is determined, the determined Optimize the fixed-point super-parameters to obtain the fixed-point super-parameters that are actually used to fix the weights of the to-be-fixed units (referred to as the final fixed-point super-parameters in this article) to further improve the fixed-pointing of the neural network performance.

Exemplarily, based on the determined fixed-pointing super-parameters of the weights of the to-be-fixed units, the range of the final fixed-pointing super-parameters used to search for the weights of the to-be-fixed units (referred to as the second fixed-pointing in this article) may be determined. Hyperparameter candidate value range), and determine the final fixed-point hyperparameter of the weight of the to-be-fixed unit from the second fixed-pointed super-parameter candidate value range, and further, the final fixed-point based on the weight of the to-be-fixed unit The super-parameters are used to fix the weights of the units to be fixed.

In an example, as shown in FIG. 5, in step S420, determining the final fixed super parameter of the weight of the to-be-fixed unit from the second fixed super parameter candidate value range may include the following steps: step S421 , Respectively determine the output error of the neural network corresponding to each candidate value in the second fixed-point hyperparameter candidate value range; step S422, determine the candidate value with the smallest output error of the corresponding neural network as the weight of the to-be-fixed unit The final fixed-point hyperparameter of the value.

Exemplarily, the fixed-point hyperparameter of the weight of the fixed-point unit can be optimized based on the output error of the neural network, and the fixed-point super-parameter that minimizes the output error of the neural network can be selected as the final fixed point of the weight of the to-be-fixed unit化超参.

When the second fixed-point hyperparameter candidate value range is determined, the output error of the neural network corresponding to each candidate value in the second fixed-point hyperparameter candidate value range can be determined separately, and the candidate with the smallest output error of the corresponding neural network can be determined The value is determined as the final fixed-pointing hyperparameter of the weight of the to-be-fixed unit.

In an example, in step S421, respectively determining the output error of the neural network corresponding to each candidate value in the second fixed-point hyperparameter candidate value range may include: for any candidate value, based on the third type feature value and the first The four types of feature values determine the output error of the neural network corresponding to the candidate value.

Exemplarily, considering the neural network in the original floating-point state, the fixed-point processing of the neural network may bring errors to the output accuracy of the neural network, and the output of the fixed-point neural network is different from the original floating-point state. The smaller the output error of the neural network, the higher the output accuracy of the fixed-point neural network. Therefore, the fixed-point can be fixed based on the error between the output of the fixed-point neural network and the output of the neural network in the original floating-point state Super parameters are optimized.

For any candidate value in the second fixed-point hyperparameter candidate value range, the eigenvalues output by each unit of the neural network when the weight of the fixed-point unit is fixed using the candidate value (referred to herein as The third type feature value), and, in the original floating point state, the feature value output by each unit of the neural network (herein referred to as the fourth type feature value), determine the output error of the neural network corresponding to the candidate value, and its specific implementation It may be described below in conjunction with specific examples, and the embodiments of the present application will not be repeated here.

In a possible embodiment, in step S100, for the to-be-fixed unit of the neural network, based on the data distribution of the feature value output by the to-be-fixed unit under the condition that the fixed unit of the neural network remains fixed, Before determining the fixed-pointing hyperparameter of the feature value output by the to-be-fixed unit, the neural network fixed-pointing method provided in the embodiment of the present application may further include: the pending point is based on the situation that the fixed-pointed unit of the neural network maintains the fixed-pointing state The eigenvalues output by the transformation unit, and the eigenvalues output by the to-be-fixed unit in the original floating-point state, optimize the weight of the to-be-fixed unit.

Exemplarily, considering that each layer of the neural network is fixed-pointed, there may be errors in accuracy. As the number of layers deepens, the cumulative error in fixed-pointing may bring a greater impact on the overall performance of the neural network. Therefore, reducing the deviation caused by the fixed-pointing of each layer can reduce the accuracy loss of each layer of the neural network after the fixed-pointing, and improve the overall performance of the neural network.

Exemplarily, for the to-be-fixed unit of the neural network, before the eigenvalues output by the to-be-fixed unit are fixed in the manner described in the above embodiment, the fixed-point unit based on the neural network may be kept fixed. The eigenvalue output by the to-be-fixed unit in the state of the state and the characteristic value output by the to-be-fixed unit in the original floating-point state are optimized for the weight of the to-be-fixed unit.

When the to-be-fixed unit is the first to-be-fixed unit of the neural network, since there is no fixed-point unit before the to-be-fixed unit, the weight optimization may not be performed for the first to-be-fixed unit of the neural network handle.

In one example, the above-mentioned neural network-based fixed-point unit maintains the fixed-point state of the characteristic value output by the to-be-fixed unit and the characteristic value output by the to-be-fixed unit in the original floating-point state. Optimizing the weight of the to-be-fixed unit may include: optimizing the weight of the to-be-fixed unit, so that the fixed-point unit of the neural network maintains the fixed-point state of the characteristic value output by the to-be-fixed unit , And, the error between the eigenvalues output by the to-be-fixed unit in the original floating-point state is the smallest.

Exemplarily, one may be determined based on the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network remains in the fixed-point state, and the characteristic value output by the to-be-fixed unit in the original floating-point state. The weight value of the unit to be fixed is optimized based on the principle of minimizing the error. The specific implementation can be described below in conjunction with specific examples, and the embodiments of the present application will not be repeated here.

Please refer to FIG. 6, which is a schematic flowchart of a neural network fixed-pointing method provided by an embodiment of this application. As shown in FIG. 6, the neural network fixed-pointing method may include the following steps S600 and S610.

Step S600: For the to-be-fixed unit of the neural network, the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, and the output of the to-be-fixed unit in the original floating-point state The eigenvalues of, optimize the weight of the unit to be fixed.

In the embodiments of the present application, in order to improve the flexibility of neural network fixed-pointing, the neural network can be divided into multiple units according to one layer or continuous multiple layers of one unit, and the neural network can be fixed-pointed unit by unit.

Considering that after each layer of the neural network is fixed-pointed, there may be errors in accuracy. As the number of layers deepens, the cumulative error in fixed-pointing may have a greater impact on the overall performance of the neural network, so , To reduce the deviation caused by the fixed point of each layer, can reduce the accuracy loss of each layer of the neural network after the fixed point, and improve the overall performance of the neural network.

Exemplarily, based on the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, and the characteristic value output by the to-be-fixed unit in the original floating-point state, the The weights of the pointized units are optimized.

For the first to-be-fixed unit of the neural network, since there is no fixed-point unit before the to-be-fixed unit, the optimization processing of the weight may not be performed.

In step S610, the optimized unit to be fixed is fixed.

In the embodiment of the present application, when the weight of the unit to be fixed is optimized in the manner described in step S600, the optimized unit to be fixed can be fixed.

It can be seen that in the method flow shown in Figure 6, the neural network is divided into multiple units and fixed-point processing is performed unit by unit, which improves the flexibility of the fixed-point neural network; in addition, when the unit to be fixed is fixed-pointed Previously, the weight of the fixed-point unit was optimized to reduce the error caused by the fixed-pointization of the previous fixed-point unit, reduce the accuracy loss of the neural network fixed-point, and improve the performance of the neural network fixed-point.

In a possible embodiment, in step S600, the characteristic value output by the to-be-fixed unit under the condition that the neural network-based fixed-point unit remains in the fixed-point state, and the to-be-fixed unit in the original floating-point state The output feature value, optimizing the weight of the to-be-fixed unit may include: optimizing the weight of the to-be-fixed unit, so that the fixed-point unit of the neural network maintains the fixed-point state. The error between the characteristic value output by the initiation unit and the characteristic value output by the to-be-fixed initiation unit in the original floating-point state is the smallest.

Exemplarily, one may be determined based on the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network remains in the fixed-point state, and the characteristic value output by the to-be-fixed unit in the original floating-point state. To optimize the weight of the unit to be fixed based on the principle of minimizing the error between the two, the specific implementation can be described below in conjunction with specific examples, and the embodiments of the present application will not repeat them here.

In a possible embodiment, in step S610, the optimization of the to-be-fixed unit may include: output of the to-be-fixed unit under the condition that the fixed-point unit based on the neural network maintains the fixed-point state According to the data distribution of the characteristic value, the optimized fixed-point hyperparameter of the characteristic value output by the to-be-fixed unit is determined, and based on the fixed-point super-parameter, the optimized characteristic value output by the to-be-fixed unit is fixed.

Exemplarily, considering that a certain number of sample pictures are input to the original model at one time to realize the fixed point of the output feature value of each layer, after the previous layer is fixed, for the latter layer, it The input is still the output in the floating-point state of the previous layer, and the authenticity of the data distribution is poor.

In order to obtain a more realistic data distribution of the eigenvalues output by each layer of the neural network, and to improve the performance of the neural network fixed point, you can follow the order of each layer of the neural network from front to back, with one layer or continuous multiple layers as a unit, Divide the neural network into multiple units, and fix the neural network unit by unit. When a unit of the neural network is fixed, keep the unit before the unit (that is, the unit to be fixed) (that is, it has been fixed) The unit) is in a fixed-point state, and based on the data distribution of the feature value output by the to-be-fixed unit under the condition that the fixed-point unit remains in the fixed-point state, the fixed-point hyperparameter of the feature value output by the to-be-fixed unit is determined.

In the embodiment of the present application, when the unit to be fixed is the first unit to be fixed in the neural network, for example, the first unit of the neural network, the neural network does not include the fixed unit, in this case, it can be based on the The data distribution of the characteristic value output by the to-be-fixed unit determines the fixed-point super-parameter of the characteristic value output by the to-be-fixed unit, and fixes the characteristic value output by the to-be-fixed unit based on the fixed super-parameter.

When the fixed-point hyperparameter of the characteristic value output by the optimized unit to be fixed is determined, the characteristic value output by the optimized unit to be fixed can be fixed based on the determined super-parameter.

In an example, after determining the optimized fixed-pointing hyperparameters of the characteristic values output by the to-be-fixed unit, it may further include: determining the fixed-pointing hyperparameters based on the optimized characteristic values output by the to-be-fixed unit, The third fixed-point super-parameter candidate value range; the final fixed-point super-parameter of the characteristic value output by the to-be-fixed unit after optimization is determined from the third fixed-point super-parameter candidate value range. Correspondingly, the above-mentioned fixed-pointing based on the fixed-pointing super-parameters on the optimized eigenvalues output by the to-be-fixed unit may include: a final fixed-pointing super-parameter pair optimized based on the eigenvalues output by the to-be-fixed unit The eigenvalue output by the to-be-fixed unit is fixed-pointed.

Exemplarily, when the fixed-point super-parameter of the characteristic value output by the optimized to-be-fixed unit is determined, the determined fixed-point super-parameter can also be optimized to obtain the to-be-fixed point that is actually used for optimization. The eigenvalues output by the transformation unit are used for fixed-point fixed-point super-parameters (referred to as final fixed-point super-parameters in this article) to further improve the performance of neural network fixed-point.

Exemplarily, based on the determined fixed-point super-parameters of the characteristic values output by the optimized to-be-fixed unit, the range for searching the final fixed-point super-parameters (referred to herein as the third fixed-point super-parameter candidate value) may be determined. Range), and determine the final fixed-point super-parameter of the characteristic value output by the optimized to-be-fixed unit from the third fixed-point super-parameter candidate value range.

In an example, the foregoing determination of the optimized final fixed-pointing hyperparameters of the characteristic values output by the to-be-fixed unit after optimization from the third fixed-pointing hyperparameter candidate value range may include: respectively determining the third fixed-pointing hyperparameters The output error of the neural network corresponding to each candidate value in the candidate value range; the candidate value with the smallest output error of the corresponding neural network is determined as the final fixed-pointing hyperparameter of the characteristic value output by the optimized point-to-fixed unit.

Exemplarily, the fixed-point hyperparameters of the characteristic value output by the fixed-point unit can be optimized based on the output error of the neural network, and the fixed-point super-parameter that minimizes the output error of the neural network can be selected as the output of the optimized to-be-fixed unit The final fixed-point hyperparameter of the eigenvalue.

When the third fixed-point hyperparameter candidate value range is determined, the output error of the neural network corresponding to each candidate value in the third fixed-point hyperparameter candidate value range can be determined separately, and the candidate with the smallest output error of the corresponding neural network can be determined The value is determined as the final fixed-pointing hyperparameter of the eigenvalue output by the pending-pointing unit after optimization.

In an example, the foregoing determination of the output error of the neural network corresponding to each candidate value in the third fixed-point hyperparameter candidate value range may include: for any candidate value, based on the fifth-type feature value and the sixth-type feature The value determines the output error of the neural network corresponding to the candidate value.

Exemplarily, considering the neural network in the original floating-point state, the fixed-point processing of the neural network may bring errors to the output accuracy of the neural network, and the output of the fixed-point neural network is different from the original floating-point state. The smaller the output error of the neural network, the higher the output accuracy of the fixed-point neural network. Therefore, the fixed-point can be fixed based on the error between the output of the fixed-point neural network and the output of the neural network in the original floating-point state Super parameters are optimized.

For any candidate value in the third fixed-point hyperparameter candidate value range, the characteristic value output by each unit of the neural network ( This article is called the fifth type feature value), and, in the original floating point state, the feature value output by each unit of the neural network (herein referred to as the six type feature value), determine the output error of the neural network corresponding to the candidate value, The specific implementation can be described below in conjunction with specific examples, and the embodiments of the present application will not be repeated here.

In the embodiment of the present application, for the to-be-fixed unit of the neural network, in addition to the characteristic value output by the to-be-fixed unit, the weight of the to-be-fixed unit needs to be fixed. Since the weight of the unit to be fixed is not affected by the fixed unit of the neural network, after optimizing the weight of the unit to be fixed according to the method shown in Figure 6, you can refer to the traditional neural network fixed-point solution In the implementation of fixed-point weighting, the weights of the optimized units to be fixed-point are fixed-pointed.

Exemplarily, for the to-be-fixed unit of the neural network, when the weighted fixed-point hyperparameter is determined based on the optimized data distribution of the weight of the to-be-fixed unit, the determined fixed-point hyperparameter can also be determined. Carry out optimization to determine the final fixed-point hyperparameters.

In an example, the optimization of the to-be-fixed unit may include: determining the optimized weight of the to-be-fixed unit based on the data distribution of the weight of the to-be-fixed unit after optimization. Fixed-point super-parameter; based on the optimized fixed-point super-parameter of the weight of the unit to be fixed, determine the fourth fixed-point super-parameter candidate value range; determine the optimized value from the fourth fixed-point super-parameter candidate value range The final fixed-pointing super parameter of the weight of the to-be-fixed unit; based on the optimized final fixed-pointing super-parameter of the weight of the to-be-fixed unit, the optimized weight of the to-be-fixed unit is fixed.

For the specific implementation of each step in this example, reference may be made to the relevant description in the foregoing method embodiment, and details are not described in the embodiment of the present application.

Please refer to FIG. 7, which is a schematic flowchart of a neural network fixed-pointing method provided by an embodiment of this application. As shown in FIG. 7, the neural network fixed-pointing method may include the following steps S700, S710, S720, and S730.

Step S700: Analyze the topological structure of the input floating-point model to generate a neural network data flow graph.

Step S710: Generate a data layer of the neural network based on the sample picture information in the configuration file.

Step S720: Based on the optimization strategy of the platform to be deployed in the configuration file, split and merge the topology of the neural network to obtain a preprocessed floating-point model.

In the embodiments of this application, considering the complexity of the neural network model structure under different applications and the limitations of deploying applications on different hardware platforms from the completion of floating-point training of a neural network model to deployment on the actual hardware platform Demand, in the traditional neural network fixed-point solution, it often requires a lot of human involvement in the process to ensure the correct fixed-point operation. At present, the publicly available neural network fixed-point tools, such as Neural Network Distiller or PocketFlow, have integrated more new algorithms from academia and are suitable for academic research, but the overall framework and process have poor versatility. Both tasks and models require more manual changes, which require high application personnel and reduce the efficiency of algorithm application.

In order to reduce human participation in the process of neural network positioning, embodiments of the present application provide a solution for automatically realizing neural network positioning.

Exemplarily, for the input to-be-fixed point model (floating point model), the topological structure of the floating point model can be analyzed to generate a data flow diagram of the neural network, that is, according to the process of data from input to output, sequentially The structure diagram of the floating-point model determined by each layer of the floating-point model is passed, and the data layer of the neural network is generated according to the sample image information in the configuration file, such as the access path of the sample image and the preprocessing method of the sample image.

Exemplarily, considering the platform characteristics of different deployment platforms, in order to optimize the effect of neural network deployment on different platforms, it can be based on the platform characteristics of the platform to be deployed in the configuration file (which can be called optimization strategy), and the topology of the neural network Splitting (such as splitting part of the topological structure of the floating-point model according to the optimization strategy of the platform to be deployed) and fusion (such as merging the BN layer of the floating-point model into the pre-Conv layer), etc., to obtain the pre-processed floating The point model makes the topological structure of the pre-processed floating-point model more suitable for the platform characteristics of the platform to be deployed, improves the efficiency of the pre-processed floating-point model on hardware, and optimizes the operating effect.

Step S730: Perform fixed-point conversion on the pre-processed floating-point model.

In the embodiment of the present application, when the preprocessing of the floating-point mode is completed, the pre-processed floating-point model can be fixed-point to obtain a fixed-point model that can be used for deployment. Exemplarily, the pre-processed floating-point model may be fixed-point processed in the manner described in any of the foregoing embodiments.

In the embodiment of the present application, after the preprocessed floating-point model is fixed-point, the fixed-point simulation test can be performed on the fixed-point model to evaluate the fixed-point performance of the model. According to the preset performance evaluation methods and indicators, determine whether the performance of the fixed-point model meets the demand. If the demand is met, the model can be finally converted and deployed; if the demand is not met, the fixed-point model optimization is carried out to further Improve the performance of fixed-point models.

Exemplarily, when the fixed-point performance of the model does not meet the requirements, the model can be tuned in a more global and detailed manner.

In one example, when the original data value (annotated sample) and the corresponding true value are available, model training based on a fixed-point budget can be performed.

In another example, when the original data value and the corresponding truth value are not available, the knowledge distillation training method can be used to guide the training and tuning of the fixed-point model using the original model. When the tuning performance meets the requirements, the final fixed-point hyperparameter table and the model to be converted are output.

In the embodiments of this application, when the neural network is fixed according to the method described in the above embodiments, intelligent analysis and processing can be performed based on the neural network after the fixed point, including but not limited to computer vision processing (such as image classification, target detection) Or speech segmentation, etc.) or natural language processing.

In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, the technical solutions provided by the embodiments of the present application will be described in more detail below with reference to specific examples.

Neural network fixed-pointization is a commonly used model acceleration algorithm. By quantizing the weight of the model and the output feature value into a fixed bit number, such as 8bit, 16bit or very low 1bit, 2bit, it can be well solved in hardware The above float32 type data brings complicated multiplication and accumulation calculations and huge bandwidth. The lower the number of quantized bits of the model, the more obvious the acceleration of the model on the hardware.

However, from the completion of the floating-point training of a neural network model to the deployment on the actual hardware platform, the complexity of the neural network model structure under different applications and the restrictive requirements of deploying applications on different hardware platforms, the traditional neural network In the fixed-point scheme, it often requires more human participation in the process to ensure the correct fixed-point operation. At present, the publicly available neural network fixed-point tools, such as Neural Network Distiller or PocketFlow, have integrated many newer academia algorithms and are suitable for academic research, but the overall framework and process have poor versatility. Both tasks and models require more manual changes, which require high application personnel and reduce the efficiency of algorithm application.

In actual complex application scenarios, the performance of neural networks requires high accuracy. When the network is fixed-point, performance degradation is unavoidable. At this time, it is necessary to analyze and tune the performance problems of the fixed-point network. Traditional The fixed-point neural network solution of Neural Networks requires manual tuning, which has high requirements for application personnel and low efficiency.

In response to the above-mentioned problems, embodiments of the present application provide an automated neural network fixed-point system. As shown in FIG. 8, the automated neural network fixed-point system may include a fixed-point evaluation and tuning module and a fixed-point conversion module.

Based on the automated neural network fixed-point system shown in Figure 8, the pre-configured fixed-point configuration information (which can be provided in the form of a configuration file), the to-be-fixed model (that is, the floating-point model), and the sample set can be used to obtain The deployed fixed-point model file. The specific functions of each module are described in detail below.

1. Fixed-point evaluation and tuning module

Refer to Figure 9. The functional flowchart of the fixed-point evaluation and tuning module can be shown in Figure 9, which mainly includes the following processes: floating-point model preprocessing, statistical evaluation and optimization, and fixed-point testing and tuning. Each process will be described in detail below.

1. Floating point model preprocessing

The preprocessing process of the floating-point model mainly includes: analyzing the topological structure of the input floating-point model to generate a neural network data flow diagram; based on the sample picture information in the configuration file, generating the data layer of the neural network; based on the configuration file The optimization strategy of the platform to be deployed, splitting the topology of the neural network (such as splitting part of the topological structure of the floating-point model according to the optimization strategy of the platform to be deployed) and fusion (such as merging the BN layer of the floating-point model into Pre-Conv layer) to obtain a model that is more suitable for deployment on the platform to be deployed with a topology structure (that is, a preprocessed floating-point model). The pre-processed floating-point model obtained after the pre-processing of the floating-point model can be subjected to subsequent processing.

2. Statistical evaluation and optimization

The purpose of the fixed-point neural network is to quantify the weight of the model and the output feature value into a fixed bit value to achieve the effect of accelerating the reasoning.

Exemplarily, in this embodiment, fixed-point algorithms including but not limited to linear symmetric fixed-point algorithms, linear asymmetric fixed-point algorithms, and Power-of-Two fixed-point algorithms can be used to realize neural network fixed-pointization.

The following takes linear symmetric fixed-point algorithm, linear asymmetric fixed-point algorithm, and Power-of-Two fixed-point algorithm as examples to illustrate the fixed-point hyperparameter calculation process in the traditional neural network fixed-point scheme.

For linear symmetric fixed-point algorithm, the calculation of fixed-point hyperparameters can be realized by formulas (1) and (2):

X _C ＝max(-C _x , min(X, C _x )) (1)

In formulas (1) and (2), X represents the original floating-point data, C _x represents the cut-off value of the absolute value of the floating-point data, B _w represents the fixed-point bit width, and X _Q represents the fixed-point value.

For linear asymmetric fixed-point algorithms, the calculation of fixed-point hyperparameters can be achieved by formulas (3) and (4):

X _C ＝max(C _n ,min(X,C _p )) (3)

In formulas (3) and (4), X represents the original floating point data, C _n represents the left side truncation value of the floating point data, C _p represents the right side truncation value of the floating point data, and B _w represents the bit width of the fixed point. X _Q represents the fixed-point value.

For the Power-of-Two fixed-point algorithm, the calculation of fixed-point hyperparameters can be realized by formulas (5), (6) and (7):

X _C ＝max(-C _x , min(X, C _x )) (5)

表示定点化等级集合，

是一个映射函数，用于确定

中与X _C最接近的数。 In formulas (5), (6) and (7), X represents the original floating-point data, C _x represents the cutoff value of the absolute value of the floating-point data, B _w represents the bit width of the _{fixed-point, and X Q} represents the fixed-point Value,

Represents a set of fixed-point levels,

Is a mapping function used to determine

The number that is closest to X _C.

In the above fixed-point algorithm, the cut-off value of floating-point data (that is, the fixed-point hyperparameter) needs to be used. The selection of the cut-off value is very important for the effect of fixed-point, and it is often determined according to the distribution of the original data.

In the traditional neural network fixed-point solution, the weights are fixed: for the neural network of the original floating-point state, the weight of the floating-point state of each unit is determined when the neural network is trained, and the floating-point state of each unit The data distribution of the weights of is not affected by the fixed-pointing of the previous unit. Therefore, the data distributions of the weights of each unit of the neural network in the original floating-point state can be separately counted, and the fixed-point hyperparameters can be calculated.

For the fixed point of eigenvalues: in order to obtain the data distribution of the eigenvalues output by each layer of the neural network, a certain number of samples need to be input to the model for forward calculation.

In the traditional neural network fixed point solution, in order to realize the fixed point of the output feature value, a certain number of sample pictures are input to the original model at one time, and the data distribution of the output feature value of each layer is recorded through the original model forward calculation Next, calculate the fixed-point hyperparameters of the eigenvalues output by each layer respectively.

In the embodiments of this application, considering that in the traditional neural network fixed-point solution, the calculation of the fixed-point hyperparameters of the eigenvalues output by each layer of the neural network is performed under the floating-point model, and the fixed-point of the pre-layer is not considered. The influence of post-modification on the post-layer causes the data distribution of the output characteristic values to deviate from the actual situation, which does not conform to the real data distribution in the fixed-point state.

Based on this, in the process of statistical evaluation and optimization, the neural network fixed-point solution provided by the embodiments of the present application has at least the following improvements.

2.1, layer by layer evaluation

In order to obtain a more realistic data distribution of the eigenvalues output by each layer of the neural network, a layer-by-layer evaluation (that is, a unit including a layer of the neural network as an example) can be used to calculate the fixed-point hyperparameters, and the current layer ( To be fixed-pointed layer) during evaluation, all layers before the current layer remain fixed-pointed state, and all layers after the current layer keep floating-point state.

In the following, the linear symmetric fixed-point algorithm, the linear asymmetric fixed-point algorithm, and the Power-of-Two fixed-point algorithm are still taken as examples to describe the fixed-point hyperparameter calculation implemented by the layer-by-layer evaluation method in the embodiment of the present application.

For the linear symmetric fixed-point algorithm, in the floating-point operation state, input a certain number of sample pictures to the original model for forward calculation, and calculate the data distribution of the eigenvalues and weights output by the first to-be-fixed layer. The fixed-point super-parameters of the eigenvalues and the fixed-point super-parameters of the weights output by the inscription layer, and use formulas (1) and (2) to convert the to-be-fixed layer into a fixed-point operation state.

For any subsequent to-be-fixed layer, while the fixed-point layer remains in the fixed-point state, again input a certain number of sample pictures to the model for forward calculation, and count the feature values and weights output by the to-be-fixed layer The data distribution of the feature, calculate the fixed super parameter of the feature value output by the to-be-fixed layer and the fixed super-parameter of the weight value, and use the formulas (1) and (2) to convert the to-be-fixed layer into a fixed-point Operation state, until all layers of the model are converted into fixed-point operation state.

For the linear asymmetric fixed-point algorithm, in the floating-point operation state, input a certain number of sample images to the original model for forward calculation, and calculate the data distribution of the eigenvalues and weights output by the first to-be-fixed layer The fixed-point super-parameters of the feature values and the fixed-point super-parameters of the weights output by the to-be-fixed layer, and use formulas (3) and (4) to convert the to-be-fixed layer into a fixed-point operation state.

For any subsequent to-be-fixed layer, while the fixed-point layer remains in the fixed-point state, again input a certain number of sample pictures to the model for forward calculation, and count the feature values and weights output by the to-be-fixed layer The data distribution of the feature, calculate the fixed super-parameter of the feature value output by the to-be-fixed layer and the fixed-point super-parameter of the weight value, and use the formulas (3) and (4) to convert the to-be-fixed layer into a fixed-point Operation state, until all layers of the model are converted into fixed-point operation state.

For the Power-of-Two fixed-point algorithm, in the floating-point operation state, a certain number of sample images are input to the original model for forward calculation, and the data distribution of the eigenvalues and weights output by the first to-be-fixed layer is counted. Calculate the fixed-point super-parameters of the eigenvalues output by the to-be-fixed layer and the fixed-point super-parameters of the weights, and use formulas (5), (6) and (7) to convert the to-be-fixed layer into a fixed-point operation state.

For any subsequent to-be-fixed layer, while the fixed-point layer remains in the fixed-point state, again input a certain number of sample pictures to the model for forward calculation, and count the feature values and weights output by the to-be-fixed layer The data distribution of the feature, calculate the fixed super parameter of the feature value output by the to-be-fixed layer and the fixed super-parameter of the weight value, and use the formulas (5), (6) and (7) to convert the to-be-fixed layer It is a fixed-point computing state until all layers of the model are converted into a fixed-point computing state.

2.2, fixed-point hyperparameter optimization

In the process of layer-by-layer fixed-point hyperparameter evaluation, the determined fixed-point hyperparameters can also be optimized, that is, after the fixed-point hyperparameters (including the fixed-point hyperparameters and/or output eigenvalues of the output eigenvalues) are determined according to the above method When the weighted fixed-point super-parameters), the search range of the final fixed-point super-parameters can be determined based on the determined fixed-point super-parameters (such as the first fixed-point super-parameter candidate value range or the second fixed-point super-parameter above). Candidate value range, etc.), and search for the fixed-point hyperparameter that has the least impact on the output error of the neural network in the search range, as the final fixed-point hyperparameter.

Taking the linear symmetric fixed-point algorithm as an example, first determine _{the search range of the fixed-point hyperparameter c w} , and then perform a traversal search within this range, calculate _{the error of the neural network output under different c w} , and select the output of the neural network The value with the least influence of error is used as the final fixed-point hyperparameter.

Exemplarily, the calculation method of the neural network output error includes, but is not limited to, the mean square error, KL divergence, Euclidean distance, and the like. Taking the mean square error as an example, the output error of the neural network can be determined in the following way:

表示原始浮点状态下该层的输出特征值，

表示均方误差函数，|O _i|表示特征数据的数量。 In formulas (8) and (9), L represents the total number of layers of the model, O _i represents the output feature value of the layer in the fixed-point state,

Represents the output feature value of the layer in the original floating point state,

Represents the mean square error function, and |O _i | represents the number of feature data.

Through formula (8), the output error of layer i of the neural network can be determined. Based on the output error of each layer of the neural network, the output error of the neural network can be determined by formula (9).

2.3. Optimization strategies to minimize local errors

Considering that after each layer of the neural network is fixed-pointed, there may be errors in accuracy. As the number of layers deepens, the cumulative error in fixed-pointing may have a greater impact on the overall performance of the neural network, so , To reduce the deviation caused by the fixed point of each layer, can reduce the accuracy loss of each layer of the neural network after the fixed point, and improve the overall performance of the neural network.

Exemplarily, based on the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, and the characteristic value output by the to-be-fixed unit in the original floating-point state, the The weights of the pointization unit are optimized, and the specific implementation process may include the following steps 1 and 2.

_{Step 1. Perform a fixed-point evaluation on the first} to-be-fixed layer L 1 of the original model M to obtain a fixed-point hyperparameter, and then fix the layer to obtain a model Q ₁ .

Step 2, the calculation model and the original model M L Q ₁ in the first floating point error output layer L ₂ after _a layer to minimize this error to optimize the power of _two L principle.

Exemplarily, determining the output error of the original model and the quantized model in the to-be-fixed layer can be achieved through functions such as L1-loss, L2-loss, and expected deviation. Taking L2-loss as an example, the optimization function is:

代表L ₂层的前向推理函数，

和

分别表示L ₂层的权值和bias(以L ₂层为卷积层为例)，Y为第一步中从原始模型M中第L ₂层提取的特征值，X _Q为从当前模型Q ₁中 第L ₂层的输入特征值，

和

分别表示L ₂层优化后的权值和bias(此时仍然浮点状态)。 In formulas (10) and (11),

Represents the forward inference function of the _{L 2 layer,}

and

L ₂ represent the weight of the layer and the BIAS (L ₂ layer to an example convolution layer), Y is the first step extracted from the original model M eigenvalues layer L _2, X _Q Q from the current model The input eigenvalues of the L _{2 layer in 1,}

and

They respectively represent the _{optimized weight and bias of the L 2} layer (still floating-point at this time).

和

)赋值给L ₂层，这样就完成了局部误差的修正，可以缓解模型定点化带来的误差。 By optimizing weights and BIAS layer L _2, Q ₁ so that the model output characteristic of the output characteristic value of the layer L ₂ of the original model M value Y of the second layer L ₂ as close as possible. The new weight and bias obtained after tuning (that is, the above

and

) Is assigned to the L ₂ layer, which completes the correction of the local error, which can alleviate the error caused by the fixed point of the model.

Repeat the above two steps until the eigenvalues and weights of the last layer in the neural network are fixed, and the model statistical evaluation and optimization process is completed. The schematic diagram can be seen in Figure 10.

As shown in Figure 10, the solid-line box represents the original floating-point layer, the dashed box represents the fixed-point layer obtained by fixed-pointing the original floating-point layer, and the solid-line box with filled squares represents the updated floating-point layer based on minimizing local errors. For the point layer, the dashed frame with filled squares represents the fixed-point layer obtained after the updated floating-point layer is fixed-point.

In the embodiment of the present application, due to the output error between the eigenvalues output by each layer of the neural network under the original floating-point model and the eigenvalue output under the quantization model, the performance of the fixed-point neural network is evaluated instead of According to the error between the final output of the quantized model and the true value of the sample in the traditional neural network fixed-point solution, the performance of the neural network fixed-point is evaluated. Therefore, the sample used for the neural network fixed-point can be without annotated information The sample, which reduces the requirements on the sample, and improves the feasibility of the plan.

3. Fixed-point testing and tuning

After the pre-processed floating-point model is fixed-point, the fixed-point model can be subjected to a fixed-point simulation test to evaluate the fixed-point performance of the model. According to the preset performance evaluation methods and indicators, determine whether the performance of the fixed-point model meets the demand. If the demand is met, the model can be finally converted and deployed; if the demand is not met, the fixed-point model optimization is carried out to further Improve the performance of fixed-point models.

Exemplarily, when the fixed-point performance of the model does not meet the requirements, the model can be tuned in a more global and detailed manner.

In one example, when the original data value (annotated sample) and the corresponding true value are available, model training based on a fixed-point budget can be performed.

In another example, when the original data value and the corresponding truth value are not available, the knowledge distillation training method can be used to guide the training and tuning of the fixed-point model using the original model. When the tuning performance meets the requirements, the final fixed-point hyperparameter table and the model to be converted are output.

2. Fixed-point conversion module

Based on the fixed-point hyperparameter table and the model to be converted output by the fixed-point evaluation and tuning module, the fixed-point conversion module can output a deployable file to deploy a quantified neural network model on the platform to be deployed based on the deployable file.

When the neural network is fixed, intelligent analysis and processing can be performed based on the fixed neural network, including but not limited to computer vision processing (such as image classification, target detection, or speech segmentation, etc.) or natural language processing.

It can be seen from the above description that in the technical solution provided by the embodiments of the present application, for the to-be-fixed unit of the neural network, the characteristics of the output of the to-be-fixed unit based on the situation that the fixed-point unit of the neural network maintains the fixed-point state The data distribution of the value, the fixed-point hyperparameter of the eigenvalue output by the to-be-fixed unit is determined, and the eigenvalue output by the to-be-fixed unit is fixed based on the fixed-point super-parameter, and the neural network is unit-by-unit. Fixed-point, for the to-be-fixed unit of the neural network, keep the fixed-point unit in the fixed-point state to obtain a more realistic data distribution of the output feature value, thereby improving the performance of the fixed-point neural network.

The method provided by this application is described above. The device provided by this application is described below.

Please refer to FIG. 11, which is a schematic structural diagram of a neural network fixed-pointing device provided by an embodiment of this application. As shown in FIG. 11, the neural network fixed-pointing device may include: a determination module for determining a pending unit of the neural network , Based on the data distribution of the feature value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, the fixed-point hyperparameter of the feature value output by the to-be-fixed unit is determined; the fixed-point module , Used to fix the characteristic value output by the unit to be fixed based on the fixed super parameter.

In a possible embodiment, after determining the fixed-pointing hyperparameter of the characteristic value output by the to-be-fixed unit, the determining module is further configured to: determine the fixed-pointing hyperparameter based on the characteristic value output by the to-be-fixed unit The first fixed point hyperparameter candidate value range; the final fixed point hyperparameter of the characteristic value output by the to-be-fixed unit is determined from the first fixed point hyperparameter candidate value range. Correspondingly, the fixed-pointing module, based on the fixed-pointing super-parameter, fixes the characteristic value output by the to-be-fixed unit, including: the final fixed-pointing super-parameter based on the characteristic value output by the to-be-fixed unit to the pending point The eigenvalues output by the transformation unit are fixed-pointed.

In a possible embodiment, the determining module determines the final fixed-pointing hyper-parameter of the characteristic value output by the to-be-fixed unit from the range of the first fixed-pointing super-parameter candidate value, including: separately determining the first fixed-pointing hyperparameter The output error of the neural network corresponding to each candidate value in the fixed-point hyperparameter candidate value range; the candidate value with the smallest output error of the corresponding neural network is determined as the final fixed-point hyperparameter of the feature value output by the to-be-fixed unit .

In a possible embodiment, the determining module separately determines the output error of the neural network corresponding to each candidate value in the first fixed-point hyperparameter candidate value range, including: for any candidate value, based on the first type The feature value and the second type feature value determine the output error of the neural network corresponding to the candidate value, and the first type feature value is the neural network when the feature value output by the to-be-fixed unit is fixed using the candidate value The feature value output by each unit; the second type feature value is the feature value output by each unit of the neural network in the original floating point state.

In a possible embodiment, the determining module is further configured to determine the weight of the pending unit of the neural network based on the data distribution of the weight of the pending unit The fixed-point super-parameter of the; based on the fixed-point super-parameter of the weight of the to-be-fixed unit, determine the second fixed-point super-parameter candidate value range; determine the to-be-fixed unit from the second fixed-point super-parameter candidate value range The final fixed-point hyperparameters of the weights. Correspondingly, the fixed-pointing module is further configured to fix the weight of the to-be-fixed unit based on the final fixed-pointing hyperparameter of the weight of the to-be-fixed unit.

In a possible embodiment, the determining module determines the final fixed-pointing hyperparameter of the weight value of the to-be-fixed unit from the second fixed-pointing super-parameter candidate value range, including: separately determining the second fixed-pointing hyperparameter The output error of the neural network corresponding to each candidate value in the hyperparameter candidate value range; the candidate value with the smallest output error of the corresponding neural network is determined as the final fixed point super parameter of the weight of the to-be-fixed unit.

In a possible embodiment, the determining module separately determines the output error of the neural network corresponding to each candidate value in the second fixed-point hyperparameter candidate value range, including: for any candidate value, based on a third type of feature Value and the fourth type feature value determine the output error of the neural network corresponding to the candidate value. When the third type feature value uses the candidate value to fix the weight of the to-be-fixed unit, each of the neural networks The feature value output by the unit; the fourth type feature value is the feature value output by each unit of the neural network in the original floating point state.

In a possible embodiment, for the to-be-fixed unit of the neural network, the determining module is based on the feature value output by the to-be-fixed unit in the case where the fixed-point unit of the neural network remains in a fixed-point state Before determining the fixed-pointing hyperparameter of the feature value output by the to-be-fixed unit, the determining module is also used to: based on the situation that the fixed-pointing unit of the neural network maintains the fixed-pointing state, the pending-pointing The eigenvalue output by the unit and the eigenvalue output by the to-be-fixed unit in the original floating-point state are optimized for the weight of the to-be-fixed unit.

In a possible embodiment, the determining module is based on the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, and the to-be-fixed unit in the original floating-point state The output characteristic value, optimizing the weight of the pending unit, includes: optimizing the weight of the pending unit, so that the determined unit of the neural network maintains the fixed state when the pending unit The error between the characteristic value output by the initiation unit and the characteristic value output by the to-be-fixed initiation unit in the original floating-point state is the smallest.

In a possible embodiment, the unit includes one layer or successive multiple layers of the neural network, and the neural network includes a plurality of units.

Please refer to FIG. 12, which is a schematic structural diagram of a neural network fixed-pointing device provided by an embodiment of this application. As shown in FIG. 12, the neural network fixed-pointing device may include: an optimization module for undetermined neural network Unit, based on the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, and the characteristic value output by the to-be-fixed unit in the original floating-point state, for the pending point The weight of the unit is optimized; the fixed-point module is used to fix the optimized unit to be fixed.

In a possible embodiment, the optimization module is based on the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network remains in the fixed-point state, and the to-be-fixed unit in the original floating-point state The output characteristic value, optimizing the weight of the pending unit, includes: optimizing the weight of the pending unit, so that the determined unit of the neural network maintains the fixed state when the pending unit The error between the characteristic value output by the initiation unit and the characteristic value output by the to-be-fixed initiation unit in the original floating-point state is the smallest.

In a possible embodiment, the fixed-pointing module performs fixed-pointing on the optimized unit to be fixed, including: performing the fixed-pointing based on the situation that the fixed-pointing unit of the neural network maintains the fixed-pointing state. The data distribution of the eigenvalues output by the unit, determine the optimized fixed-point hyperparameters of the eigenvalues output by the to-be-fixed unit, and fix the optimized eigenvalues output by the to-be-fixed unit based on the fixed-point hyperparameters change.

In a possible embodiment, after the fixed-pointing module determines the optimized fixed-pointing hyperparameter of the characteristic value output by the to-be-fixed unit, it is also used to: based on the optimized characteristic value output by the to-be-fixed unit Determine the third fixed-point super-parameter candidate value range; from the third fixed-point super-parameter candidate value range, determine the optimized final fixed-point super-parameter of the characteristic value output by the to-be-fixed unit. Correspondingly, the fixed-pointing module fixes the optimized eigenvalues output by the unit to be fixed based on the super-parameters, including: the final fixed-point super-parameter pair optimized based on the characteristic values output by the unit to be fixed The eigenvalues output by the to-be-fixed unit are fixed-pointed.

In a possible embodiment, the fixed-pointing module determines from the third fixed-pointing super-parameter candidate value range the final fixed-pointing super-parameters of the characteristic values output by the to-be-fixed unit after optimization, including: respectively determining The output error of the neural network corresponding to each candidate value in the third fixed-point hyperparameter candidate value range; the candidate value with the smallest output error of the corresponding neural network is determined as the eigenvalue output of the optimized unit to be fixed Finally, the super-parameters are fixed-pointed.

In a possible embodiment, the fixed-pointing module separately determines the output error of the neural network corresponding to each candidate value in the third fixed-pointing hyperparameter candidate value range, including: for any candidate value, based on the fifth type The feature value and the sixth type feature value determine the output error of the neural network corresponding to the candidate value; wherein the fifth type feature value uses the candidate value to locate the optimized feature value output by the to-be-fixed unit When, the eigenvalues output by the units of the neural network; the sixth type eigenvalues are the eigenvalues output by the units of the neural network in the original floating-point state.

In a possible embodiment, the neural network includes a plurality of units, and each unit includes one layer or successive multiple layers of the neural network.

Please refer to FIG. 13, which is a schematic structural diagram of a neural network fixed-pointing device provided by an embodiment of this application. As shown in FIG. 13, the neural network fixed-pointing device may include: an analysis module for analyzing the input floating-point model The topology structure is analyzed to generate a neural network data flow graph; a generation module is used to generate the data layer of the neural network based on the sample picture information in the configuration file; the processing module is used to generate the data layer of the neural network based on the configuration file to be deployed The optimization strategy of the platform is to split and merge the topology of the neural network to obtain the pre-processed floating-point model; the fixed-point module is used to perform the pre-processing based on the method described in the above method embodiment The floating-point model is fixed-point.

Please refer to FIG. 14, which is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of this application. The electronic device may include a processor 1401 and a memory 1402 storing machine-executable instructions. The processor 1401 and the memory 1402 may communicate via a system bus 1403. In addition, by reading and executing the machine executable instructions corresponding to the encoding control logic in the memory 1402, the processor 1401 can execute the neural network fixed-pointing method described above.

The memory 1402 mentioned herein may be any electronic, magnetic, optical, or other physical storage device, and may contain or store information, such as executable instructions, data, and so on. For example, the machine-readable storage medium can be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard drive), solid state drive, any type of storage disk (Such as CD, DVD, etc.), or similar storage media, or a combination of them.

In some embodiments, a machine-readable storage medium is also provided, such as the memory 1402 in FIG. 14. The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are executed by the processor, Realize the neural network fixed-point method described above. For example, the machine-readable storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

In some embodiments, a computer program is also provided, the computer program is stored in a machine-readable storage medium, and when the computer program is executed by a processor, the neural network fixed-pointing method described above is realized.

In this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such existence between these entities or operations. The actual relationship or order. Moreover, the terms "including", "including" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements includes not only those elements, but also those that are not explicitly listed. Other elements, or also include elements inherent to this process, method, article, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or equipment that includes the element.

The above are only some embodiments of this application and are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection of this application. Within the range.

Claims (25)

A neural network fixed-point method, including: For the to-be-fixed unit of the neural network, based on the data distribution of the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, determine the value of the characteristic value output by the to-be-fixed unit Fixed-point hyperparameters; The characteristic value output by the unit to be fixed is fixed based on the fixed super parameter. The method of claim 1, wherein: After determining the fixed-pointing hyperparameter of the characteristic value output by the to-be-fixed-pointing unit, the method further includes: Determine the candidate value range of the first fixed-point super-parameter based on the fixed-point super-parameter of the characteristic value output by the to-be-fixed unit; Determine the final fixed-point super-parameter of the characteristic value output by the to-be-fixed unit from the first fixed-point super-parameter candidate value range; The step of positioning the characteristic value output by the unit to be fixed based on the super-parameter of the fixed point includes: The final fixed-point hyperparameter based on the characteristic value output by the to-be-fixed unit fixes the characteristic value output by the to-be-fixed unit. The method according to claim 2, wherein the determining the final fixed-pointing hyper-parameter of the characteristic value output by the to-be-fixed-pointing unit from the first fixed-pointing super-parameter candidate value range comprises: Respectively determine the output error of the neural network corresponding to each candidate value in the first fixed-point hyperparameter candidate value range; The candidate value with the smallest output error of the corresponding neural network is determined as the final fixed-point hyperparameter of the feature value output by the to-be-fixed unit. 4. The method according to claim 3, wherein said determining respectively the output error of the neural network corresponding to each candidate value in the first fixed-point hyperparameter candidate value range comprises: For any candidate value, the output error of the neural network corresponding to the candidate value is determined based on the first type feature value and the second type feature value; wherein, the first type feature value uses the candidate value for the unit to be fixed When the output feature value is fixed-point, the feature value output by each unit of the neural network; the second type feature value is the feature value output by each unit of the neural network in the original floating point state. The method according to any one of claims 1-4, wherein the method further comprises: Based on the data distribution of the weight of the to-be-fixed unit, determine the fixed-point hyperparameter of the weight of the to-be-fixed unit; Determine the second fixed-point super-parameter candidate value range based on the fixed-point super-parameter of the weight of the to-be-fixed unit; Determining the final fixed-pointing super-parameter of the weight value of the to-be-fixed unit from the second fixed-pointing super-parameter candidate value range; The final fixed-pointing hyperparameter based on the weight of the to-be-fixed unit fixes the weight of the to-be-fixed unit. The method according to claim 5, wherein the determining the final fixed-pointing hyperparameter of the weight value of the to-be-fixed unit from the second fixed-pointing hyperparameter candidate value range comprises: Respectively determine the output error of the neural network corresponding to each candidate value in the second fixed-point hyperparameter candidate value range; The candidate value with the smallest output error of the corresponding neural network is determined as the final fixed-point hyperparameter of the weight of the to-be-fixed unit. 7. The method according to claim 6, wherein said separately determining the output error of the neural network corresponding to each candidate value in the second fixed-point hyperparameter candidate value range comprises: For any candidate value, the output error of the neural network corresponding to the candidate value is determined based on the third type feature value and the fourth type feature value; wherein the third type feature value uses the candidate value for the unit to be fixed When the weight value of is fixed-point, the feature value output by each unit of the neural network; the fourth type feature value is the feature value output by each unit of the neural network in the original floating point state. The method according to any one of claims 1-7, wherein for the pending unit of the neural network, the pending unit based on the neural network maintains a fixed state when the pending unit The data distribution of the eigenvalues output by the pointization unit, before determining the fixed point hyperparameters of the eigenvalues output by the to-be-fixed unit, also includes: The eigenvalues output by the unit to be fixed when the fixed-point unit based on the neural network maintains the fixed-point state, and the characteristic value output by the unit to be fixed in the original floating-point state, to the unit to be fixed The weights are optimized. 8. The method according to claim 8, wherein the eigenvalues output by the to-be-fixed unit under the condition that the fixed-point unit based on the neural network maintains the fixed-point state, and the original floating-point state The eigenvalue output by the undetermined unit to optimize the weight of the undetermined unit includes: The weight of the to-be-fixed unit is optimized, so that the characteristic value of the to-be-fixed unit output when the fixed-point unit of the neural network remains in the fixed-point state, and the pending point in the original floating-point state The error between the eigenvalues output by the chemical unit is the smallest. The method according to any one of claims 1-9, wherein the neural network comprises a plurality of units, and each unit comprises one layer or successive multiple layers of the neural network. A neural network fixed-point method, including: For the to-be-fixed unit of the neural network, the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, and the output of the to-be-fixed unit in the original floating-point state Eigenvalue, to optimize the weight of the unit to be fixed; The optimized unit to be fixed is fixed. The method according to claim 11, wherein the eigenvalues output by the to-be-fixed unit under the condition that the fixed-point unit based on the neural network maintains the fixed-point state, and the eigenvalues output by the pending unit in the original floating-point state The eigenvalue output by the undetermined unit to optimize the weight of the undetermined unit includes: The weight of the to-be-fixed unit is optimized, so that the characteristic value of the to-be-fixed unit output when the fixed-point unit of the neural network remains in the fixed-point state, and the pending point in the original floating-point state The error between the eigenvalues output by the chemical unit is the smallest. The method according to claim 11 or 12, characterized in that, the positioning of the optimized unit to be fixed comprises: Based on the data distribution of the feature value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, determine the optimized fixed-point hyperparameter of the feature value output by the to-be-fixed unit; Based on the fixed-point hyperparameter, the optimized eigenvalue output by the to-be-fixed unit is fixed-pointed. The method of claim 13, wherein: After determining the optimized fixed-point hyperparameter of the characteristic value output by the to-be-fixed unit, the method further includes: Based on the optimized fixed-point super-parameter of the characteristic value output by the to-be-fixed unit, determine the third fixed-point super-parameter candidate value range; Determine the optimized final fixed-point super-parameter of the eigenvalue output by the to-be-fixed unit after optimization from the third fixed-point super-parameter candidate value range; The step of fixing the characteristic value output by the optimized unit to be fixed based on the fixed super parameter includes: Based on the final fixed-point hyperparameter of the characteristic value output by the to-be-fixed unit, the optimized characteristic value output by the to-be-fixed unit is fixed. 15. The method according to claim 14, wherein the determining the optimized final fixed-pointing hyper-parameter of the characteristic value output by the to-be-fixed unit after optimization from the third fixed-pointing super-parameter candidate value range comprises: Respectively determine the output error of the neural network corresponding to each candidate value in the third fixed-point hyperparameter candidate value range; The candidate value with the smallest output error of the corresponding neural network is determined as the final fixed-point hyperparameter of the eigenvalue output by the to-be-fixed unit after optimization. 15. The method according to claim 15, wherein the separately determining the output error of the neural network corresponding to each candidate value in the third fixed-point hyperparameter candidate value range comprises: For any candidate value, the output error of the neural network corresponding to the candidate value is determined based on the fifth type feature value and the sixth type feature value; wherein, the fifth type feature value is the pending value optimized using the candidate value pair When the feature value output by the pointization unit is fixed-pointed, the feature value output by each unit of the neural network; the sixth type feature value is the feature value output by each unit of the neural network in the original floating-point state. The method according to any one of claims 11-16, wherein the neural network comprises a plurality of units, and each unit comprises one layer or successive multiple layers of the neural network. The method according to any one of claims 11-17, wherein the step of fixing the optimized unit to be fixed comprises: Based on the optimized data distribution of the weight of the to-be-fixed unit, determine the optimized fixed-point hyperparameter of the weight of the to-be-fixed unit; Determine the fourth fixed-point super-parameter candidate value range based on the optimized fixed-point super-parameter of the weight of the to-be-fixed unit; Determining the optimized final fixed-pointing hyperparameter of the weight of the to-be-fixed unit after optimization from the fourth fixed-pointing super-parameter candidate value range; Based on the optimized final fixed-pointing hyperparameter of the weight of the to-be-fixed unit after optimization, the weight of the optimized to-be-fixed unit is fixed. A neural network fixed-point method, including: Analyze the topological structure of the input floating-point model to generate a neural network data flow graph; Generating the data layer of the neural network based on the sample picture information in the configuration file; Splitting and fusing the topology of the neural network based on the optimization strategy of the platform to be deployed in the configuration file to obtain a preprocessed floating-point model; The pre-processed floating-point model is fixed-point based on the method of any one of claims 1-18. A neural network fixed-point device, including: The determining module is used to determine the to-be-fixed unit of the neural network based on the data distribution of the feature value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state. Fixed-point hyperparameters of output eigenvalues; The fixed point module is used to fix the characteristic value output by the unit to be fixed based on the fixed super parameter. A neural network fixed-point device, including: The optimization module is used for the to-be-fixed unit of the neural network, based on the characteristic value output by the to-be-fixed unit under the condition that the fixed-point unit of the neural network maintains the fixed-point state, and the undetermined in the original floating-point state The eigenvalues output by the initiation unit optimize the weight of the to-be-initiated unit; The fixed point module is used to fix the optimized unit to be fixed. A neural network fixed-point device, including: The analysis module is used to analyze the topological structure of the input floating-point model to generate a neural network data flow graph; A generating module for generating the data layer of the neural network based on the sample picture information in the configuration file; The processing module is used to split and merge the topology of the neural network based on the optimization strategy of the platform to be deployed to obtain the preprocessed floating-point model; The fixed-point module is used to fix-point the pre-processed floating-point model based on the method of any one of claims 1-18. An electronic device comprising a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement any one of claims 1-18 The method described in the item. A machine-readable storage medium having machine-executable instructions stored in the machine-readable storage medium, and when the machine-executable instructions are executed by a processor, the method according to any one of claims 1-18 is implemented. A computer program, the computer program being stored in a machine-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1-18 is implemented.

Images