WO2022061867A1 - Data processing method and apparatus, and computer-readable storage medium - Google Patents

Abstract

A data processing method and apparatus, and a computer-readable storage medium. The method is applied to a processing apparatus for performing a convolution operation. The processing apparatus comprises a data selector. The method comprises: loading an input value and a weight value (S101); inputting the input value and the weight value into a data selector, and obtaining a multiplication operation result in an convolution operation by using the data selector (S102); and obtaining an operation result of the convolution operation on the basis of the multiplication operation result (S103). By means of the method, a multiplier is replaced with the data selector, and the multiplication operation result in the convolution operation is obtained by using the data selector, such that the operation complexity is reduced, thereby improving the operation speed of a neural network.

Description

Data processing method, apparatus and computer readable storage medium technical field

The present application relates to the technical field of data processing, and in particular, to a data processing method, apparatus, and computer-readable storage medium.

Background technique

With the development of technology, convolutional neural network technology is applied to all aspects of life, such as image recognition (such as face recognition, content-based image retrieval or expression recognition, etc.) using convolutional neural network technology, natural language processing (such as Speech recognition, text classification or information retrieval, etc.) and so on.

However, the operation of a convolutional neural network is a computationally and memory-intensive process. In the actual application process, the product functions are required to meet real-time requirements. For example, in the case of using convolutional neural networks for image processing, the frame rate is required to reach 10 to 20 frames per second, that is, the convolutional neural network is required to be able to Processes 10 or 20 frames per second. However, due to the operating characteristics of the convolutional neural network (intensive computation and intensive storage), its running speed is slow, often unable to achieve the required processing frame rate, unable to meet the real-time requirements of the product, and affecting the user experience.

SUMMARY OF THE INVENTION

In view of this, one of the objectives of the present application is to provide a data processing method, apparatus and computer-readable storage medium.

In a first aspect, an embodiment of the present application provides a processing device applied to a convolution operation, where the processing device includes a data selector; the method includes:

Load input values and weight values;

Inputting the input value and the weight value into the data selector, and using the data selector to obtain the multiplication result in the convolution operation;

The operation result of the convolution operation is obtained based on the multiplication operation result.

In a second aspect, an embodiment of the present application provides a data processing apparatus, the apparatus is used to perform a convolution operation, and includes a data loading module, a data selector, and an adder;

the data loading module, for loading the input value and the weight value and inputting it into the data selector;

the data selector, configured to output the multiplication result in the convolution operation according to the input value and the weight value, and send the multiplication result to the adder;

The adder is configured to output the operation result of the convolution operation according to the multiplication operation result.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer instruction is stored thereon, and when the instruction is executed by a processor, any one of the methods described in the first aspect is executed.

In a fourth aspect, an embodiment of the present application provides a movable platform, including:

body;

a power system, arranged on the body, the power system is used to provide power for the movable platform;

A data processing apparatus as described above.

In the data processing method, device, and computer-readable storage medium provided by the embodiments of the present application, in this embodiment, a data selector is used to replace the multiplier, and the data selector is used to obtain the multiplication result in the convolution operation. On the basis of the same multiplication result, the complexity of the operation is reduced, thereby improving the operation speed of the convolutional neural network and meeting the real-time requirements of actual products. In the actual application process, the convolutional neural network is used to process specific objects ( (such as images, voice signals, etc.), the processing results can be obtained faster. Correspondingly, from the user's point of view, the response speed of the product applying the convolutional neural network is faster, which is beneficial to improve the user's use. experience.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

1 is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application;

2 is a schematic diagram of obtaining a multiplication result of a convolution operation using a data selector provided by an embodiment of the present application;

3 is a schematic structural diagram of a second data processing apparatus provided by an embodiment of the present application;

4 is a schematic structural diagram of a third data processing apparatus provided by an embodiment of the present application;

5 is a schematic structural diagram of a fourth data processing apparatus provided by an embodiment of the present application;

FIG. 6 is a schematic flowchart of a data processing method provided by an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

Considering related technologies, the operation of convolutional neural network is a computationally intensive and storage-intensive process, which makes its running speed slow, often cannot meet the real-time requirements of products, and affects user experience. Among them, the convolutional neural network contains a large number of convolution operations. The purpose of the convolution operation is to extract different features of the input data. For example, in the field of image processing, different features in the image can be extracted based on the convolution operation, so that the Convolutional neural networks give better processing results based on extracted features.

Based on this, an embodiment of the present application provides a processing device, which is used to perform a convolution operation, and uses a data selector instead of a multiplier to obtain the multiplication operation result in the convolution operation, which further reduces the complexity of the operation , which is beneficial to improve the operation speed of the neural network and meet the real-time requirements of actual products.

Please refer to FIG. 1 , which is a structural diagram of a first data processing apparatus provided by an embodiment of the application. The processing apparatus is used to perform convolution operations in a neural network, and the apparatus includes a data loading module 11 and a data selector 12 and adder 13.

The data loading module 11 is used to load the input value and the weight value and input them into the data selector 12 .

Wherein, the weight value refers to a convolution kernel to perform a convolution operation on the input value. The number, size, and value of the convolution kernels may be specifically set according to the actual application scenario, which is not limited in this embodiment of the present application.

The input value refers to the data to be processed to be subjected to the convolution operation. For example, in the speech recognition scenario, the input value refers to the voice signal to be processed; for example, in the image processing scenario, the input value refers to the image to be processed. data. In an implementation manner, the input value is an activation value output by a previous layer in the convolutional neural network after operation.

The data selector 12 is configured to output the multiplication result in the convolution operation according to the input value and the weight value, and send the multiplication result to the adder 13 .

The data selector 12 is specifically configured to output the multiplication result in the convolution operation according to the input value, the inverse of the input value, and the weight value. In an implementation manner, the data selector 12 may, according to the weight value, choose to output the multiplication result in the convolution operation according to the input value, or choose to output the result according to the inverse of the input value. The result of the multiplication operation in the convolution operation. In this embodiment, the data selector 12 is used instead of the multiplier to obtain the multiplication result in the convolution operation, which further reduces the complexity of the operation, thereby improving the operation speed of the neural network and improving the user experience.

The adder 13 is configured to output the operation result of the convolution operation according to the multiplication operation result.

The adder 13 is specifically configured to perform an addition operation on the acquired multiple multiplication operation results, acquire and output the operation result of the convolution operation.

In this embodiment, the data selector 12 is used to replace the multiplier, and the data selector 12 is used to obtain the multiplication result in the convolution operation, which reduces the complexity of the operation on the basis of obtaining the same multiplication result, thereby improving the neural network. In the actual application process, when using the convolutional neural network to process specific objects (such as images, voice signals, etc.), the processing results can be obtained faster. Correspondingly, from From the user's point of view, the response speed of the product applying the convolutional neural network is faster, which is beneficial to improve the user's experience.

In one embodiment, since the operation of the convolutional neural network is a computationally intensive and memory-intensive process, the convolutional neural network occupies more bandwidth during the operation. Based on this, in order to reduce the bandwidth occupied by the convolutional neural network during operation, in this embodiment, the input value and the weight value are quantized, that is, the input value and the weight value are quantized after As a result, since the data volume of the quantized input value and the quantized weight value is reduced compared with that before quantization, when the convolution operation is performed, the quantized input value and the quantized weight value are used for the operation, and the corresponding Compared with before quantization, less computing resources are consumed, the required bandwidth is also reduced, and power consumption is reduced.

Further, considering that the 8-bit fixed-point method is usually used to deploy convolutional neural networks in related technologies, but with more and more application scenarios of convolutional neural networks, higher requirements for algorithm accuracy are put forward, so larger networks are required. The data generated by the calculation of each layer of such a network is still very large even if it is stored in 8 bits, which increases the cost of the storage device. Based on this, the present application quantizes the input value and/or (and/or represents either or both) the weight value, and the quantization bits of the input value and the weight value are both less than 8. Compared with the 8-bit fixed-point method, it can further reduce the data volume of the operation process and the storage process, thereby reducing the operation resources in the operation process and the storage resources of the stored process, reducing functions and improving performance.

It can be understood that, if the number of quantization bits of the input value and/or the weight value is less than 8, this embodiment does not impose any restrictions on the specific number of quantization bits of the input value and the weight value. , which can be specifically set according to the actual application scenario, for example, the number of quantization bits of the input value is 2, and the number of quantization bits of the weight value is 1.

In an implementation manner, when the number of quantization bits of the weight value is 1, the value of the weight value is {-1, 1}, please refer to FIG. 2 , the data selector 12 obtains the For the multiplication result in the convolution operation, the data selector 12 may select and output the input value or the opposite number of the input value according to the input weight value. In one example, when the weight value input to the data selector 12 is 1, the data selector 12 can choose to output the input value, that is, the result of the multiplication operation in the convolution operation; When the weight value of the selector 12 is -1, the data selector 12 can choose to output the opposite number of the input value, that is, the multiplication result in the convolution operation. In this embodiment, based on the data selector 12 instead of the multiplier, since the selection logic of the data selector 12 is simpler than that of the multiplier, the operation complexity of using the data selector 12 is lower than that of using the data selector 12. The operation complexity of the multiplier is beneficial to improve the operation speed of the convolutional neural network and meet the real-time requirements of actual products. In one example, a gate circuit may be used to select and output the input value or the inverse of the input value according to the weight value of the input. Regarding the data selector in this embodiment, it is not limited to processing a 1-bit weight and a 2-bit activation value (input value) or an input value of a higher bit, but can also handle an arbitrary-bit weight value and a 1-bit input value, That is, the weight value is any bit, and the input value is 1 bit. At this time, the calculation can be performed by exchanging the positions of the weight value and the input value in FIG. 2 .

Further, in order to further improve the operation speed of the convolutional neural network, the input value and/or the opposite number of the input value can be represented in complement form, then the opposite number of the input value is the inversion of the input value. The result of adding 1 afterward further simplifies the computational complexity of obtaining the opposite number of the input value, reduces the computational cost of the processing device, and thus helps to improve the computational speed of the convolutional neural network.

In one embodiment, the input value is the activation value output after the operation of the previous layer in the convolutional neural network, and the activation value quantization function in the related art defaults to the activation function that makes the activation value non-negative. Sign value quantization; in fact, activation functions used to output signed values, such as Leaky ReLU, or no activation function, still exist in some network structures, such as MobileNetV2, MNAS and other structures. Therefore, the embodiments of the present application propose a technical solution compatible with signed values and unsigned values, and the input values include signed values and unsigned values. The convolution operation is performed on unsigned values, thereby making the processing device widely applicable.

In one implementation, the signed value and the weight value can be input into the data selector 12, and the data selector 12 outputs the multiplication in the convolution operation according to the weight value and the signed value. operation result; and, the unsigned value and the weight value can be input into the data selector 12, and the data selector 12 outputs the multiplication operation in the convolution operation according to the weight value and the unsigned value result.

However, it is considered that when the data selector 12 outputs the multiplication result in the convolution operation according to the weight value and the unsigned value, overflow is likely to occur, thereby causing an error in the multiplication result. In an example, for example, the number of quantization bits of the input value is 2, the quantization value range of the signed value is {-2,-1,0,1}, and the quantization value range of the unsigned value is {0 , 1, 2, 3}; the number of quantization bits of the weight value is 1, and the quantization range is {-1, 1}. When the quantization value of the weight value is 1, the multiplication result of the signed value is 1*{-2, -1,0,1}={-2,-1,0,1}, the result of multiplication of unsigned values is 1*{0,1,2,3}={0,1 , 2, 3}; when the quantization value of the weight value is -1, the multiplication result of the signed value is -1*{-2,-1,0,1}={2,1,0, -1}; the result of multiplication of unsigned values is -1*{0, 1, 2, 3} = {0, -1, -2, -3}. In the convolution operation, the values of the convolution areas need to be summed, and the sum of the convolution areas of each input layer is accumulated. In the accumulation process, an intermediate variable with a higher bit is usually used for storage. If it is represented by 8 bits, the representation range of the value is [-128, 127], and if it is represented by 16 bits, the representation range of the value is [-32768 , 32767]. If the input is a signed value during the convolution calculation process, the 8-bit cumulative variable can tolerate 64 consecutive maximum value additions without overflow, and the 63 consecutive maximum value additions without overflow; the 16-bit cumulative variable can tolerate 16384 consecutive extremes The addition of large values does not overflow, and the addition of 16383 consecutive maximum values does not overflow. If the input is an unsigned value during the convolution calculation, the 8-bit cumulative variable can tolerate 42 consecutive maximum value additions without overflow, and the 42 consecutive maximum value additions without overflow; the 16-bit cumulative variable can tolerate 10922 consecutive maximum values The value addition does not overflow, and the addition of 10922 consecutive maximum values does not overflow. It can be seen that when an unsigned value is used for operation, since the operation result requires more bits to represent, it is more prone to overflow than a signed value.

Further, considering that the process of convolution operation in the computer is carried out in binary mode, that is, the binary representation of both signed and unsigned values is the same. In an example, it is assumed that the quantization of the input value is If the number of bits is 2, the binary form of the signed value {-2, -1, 0, 1} is represented as {00, 01, 10, 11}, and the binary form of the unsigned value {0, 1, 2, 3} Also denoted as {00, 01, 10, 11}, where a flag information is used to flag whether the input value is a signed value or an unsigned value.

Therefore, in order to solve the problem that unsigned values are more prone to overflow, and in addition, the computer performs the convolution operation in binary mode. Therefore, in the embodiment of the present application, the input values input to the data selector 12 are considered to be all The operation is performed on signed values, so to speak, the input values into the data selector 12 are signed values, thereby avoiding overflow problems.

Further, in order to ensure the accuracy of the output result, if the input value is an unsigned value (for example, it can be determined by the flag information), the adder 13, after acquiring the multiplication result, according to the multiplication The operation result and the offset value output the operation result of the convolution operation; the offset value indicates the conversion relationship between the signed value and the unsigned value. In one example, assuming that FS is a signed value, FU is an unsigned value, and a is the number of quantization bits of the input value, there is FU=FS+2 ^a-1 .

The offset value may be determined according to the number of quantization bits of the input value and the weight value. Since the number of quantized bits of the input value and the weight value are known in advance, the offset value can be obtained offline, and the offset value can be directly used in the subsequent convolution operation without the need for Repeatedly calculating the offset value is beneficial to speed up the operation speed of the convolutional neural network and the processing device.

In one embodiment, it is possible to choose whether to output a signed value or an unsigned value according to actual application requirements. If a signed value is selected to be output, an activation function such as a preLU function for outputting a signed value can be used; if an unsigned output is selected to be output value, you can use an activation function such as the reLU function for outputting unsigned values. In an example, if the number of quantization bits of the weight value is 1, the value range of the quantization value is {-1, 1}. Since the quantization value of the weight value is a signed value, according to the weight value The determined offset value is also a signed value. If the input value is an unsigned value, since the operation result of the convolution operation output by the adder 13 according to the multiplication operation result and the offset value is also a If it is a signed value, an activation function such as a reLU function for outputting an unsigned value can be used, so that the final output result is an unsigned value.

In an example, for example, the number of quantization bits of the input value is 2, the quantization value range of the signed value is {-2,-1,0,1}, and the quantization value range of the unsigned value is {0 , 1, 2, 3}; the number of quantization bits of the weight value is 1, and the quantization range is {-1, 1}. In the convolutional neural network, the activation value of the previous convolutional layer is the current convolutional layer. Input, denote the input F of the convolution layer, the size is C×H×W, the weight K, the size is N×C×k×k, the output O, the size is N×H×W, where C is the number of input channels, N is the number of output channels, k×k is the size of a single convolution kernel, and H×W is the size of the feature map, then the calculation formula of the convolution layer is as follows: O _m,l,n =∑ _i,j,c K _{i,j,c ,n} ·F _{m+i-1,l+j-1,c} (1); for signed and unsigned values, assuming FS is a signed value and FU is an unsigned value, then FU=FS+2 (2), put (2) into (1), then we have: O _m,l,n =∑ _i,j,c K _i,j,c,n ·FU _{m+i-1,l +j-1,c} =∑ _i,j,c K _i,j,c,n ·(FS _{m+i-1,l+j-1,c} +2); then O _m,l,n =∑ _i,j,c K _i,j,c,n ·FS _{m+i-1,l+j-1,c} +2∑ _i,j,c K _i,j,c,n ; If the value 2Σ _i,j,c K _{i,j,c,n is} set, the offset value can be calculated off-line, so that the operation speed of the processing device can be improved.

In an embodiment, please refer to FIG. 3 , another data processing apparatus provided by an embodiment of the present application, the processing apparatus further includes a decoder 14 .

The data loading module 11 is specifically configured to read the compressed input value and the compressed weight value from the memory.

In one implementation, the data loading module 11 may be a DMA controller (Direct Memory Access), and the DMA controller is used to read compressed input values and compressed weights from an external memory value.

The decoder 14 is used to decompress the compressed input value and the compressed weight value, respectively, and input into the data selector 12 . In this embodiment, the compressed input value and the compressed weight value are stored in the memory, which is beneficial to reduce the amount of data that needs to be stored and realize the comprehensive use of storage resources.

The data loading module 11 reads compressed input values and compressed weight values from memory and inputs them into the decoder 14, which respectively The weight value is decompressed, and the decompressed input value and the weight value are input into the data selector 12 .

In an implementation manner, both the two code streams indicating the compressed input value and the compressed weight value respectively can be obtained by encoding in a run-length encoding manner. The run-length encoding method encodes by detecting repeated sequences of bits or characters and replacing them with their occurrences.

If the number of quantization bits of the input value or the weight value is 1, the code streams corresponding to the compressed input value and the compressed weight value respectively indicate the number of consecutive quantization values; the quantization value is The quantized value of the input value and the weight value.

In one embodiment, if the number of quantization bits of the input value or the weight value is 1, in the code stream corresponding to the input value or the weight value respectively, the number of consecutive quantization values is pre-used. The set number of bits represents; the decoder 14 is specifically configured to: divide the code streams corresponding to the compressed input value or the compressed weight value respectively according to the preset number of bits, and obtain One or more sub-streams; according to the one or more sub-streams and the quantization values corresponding to the sub-streams, the decompressed input value or the decompressed weight value is obtained.

The quantization values corresponding to the sub-code streams are determined according to a preset correspondence relationship, and the correspondence relationship indicates the quantized values corresponding to different orders of the sub-code streams in the code stream. In an example, if the number of quantization bits of the input value or the weight value is 1, then the quantization range is {-1, 1}. For example, the corresponding relationship can be set to an odd number, indicating that -1 is continuous. Numbers, the order is an even number to represent the number of consecutive 1s, such as the first, the third subcode... The 2n+1 subcode stream represents the number of consecutive -1, the second, the fourth... The first The 2n sub-streams represent the number of consecutive 1s, and n is an integer; or the sequence of odd numbers represents the number of consecutive 1s, and the sequence of even numbers represents the number of consecutive -1s.

In an exemplary embodiment, if the number of quantization bits of the input value or the weight value is 1, the quantization range thereof is {-1, 1}. If the operation is performed in binary mode, 0 in binary mode can be used to represent the quantization value -1, and 1 in binary mode can be used to represent 1, then the quantization range is {-1, 1}, which is represented as {0 in binary mode, 1}. Assume that the consecutive number of each quantization value is represented by 4 bits, the odd order represents the consecutive number of 0, and the even order represents the consecutive number of 1; for example, please refer to Table 1, the original value is 32 bits, followed by 8 consecutive 0s, 12 consecutive 1s and 12 0s, the first substream after encoding is "1000", indicating the number of consecutive 0s; the second substream is "1100", indicating the number of consecutive 1s ; The third sub-stream is "1100", indicating the number of consecutive 0s; thus the encoded code stream is "100011001100", which is expressed as 8CC in hexadecimal notation, and the encoded code stream is expressed as 12 bits. , compared to the original value represented by 32 bits, the compression ratio is 62.5%, which is conducive to saving storage resources; when decoding, please refer to Table 2, the encoding is 8CC, that is, "100011001100", according to the preset 4 bits Bits can be obtained as "1000", "1100" and "1100", and then according to the preset correspondence: "The odd-numbered order represents the consecutive number of 0s, and the even-numbered order represents the consecutive number of 1s"; after decoding, they are 8 consecutive 0s, 12 consecutive 1s and 12 consecutive 0s.

Table 1

Table 2

If the number of quantization bits of the input value or the weight value is greater than 1, the code stream corresponding to the compressed input value or the compressed weight value respectively indicates the quantization value and the number of consecutive quantization values; The quantized value is a quantized value of the input value or the weight value.

In an embodiment, if the number of quantized bits of the input value or the weight value is greater than 1, in the code stream corresponding to the input value or the weight value, each quantization value and its consecutive number Use the preset number of bits to represent;

The decoder 14 is specifically configured to: divide the code stream corresponding to the compressed input value or the compressed weight value according to a preset number of bits to obtain one or more sub-code streams; For a sub-code stream, the quantized value is obtained from a specified bit of the sub-code stream and the continuous number of the quantized value is obtained from other bits. The specified bits may be specifically set according to actual application scenarios, which are not limited in this embodiment of the present application.

In an example, the number of quantization bits of the input value is 2 for illustration, and the quantization ranges of the unsigned value and the unsigned value are {-2, -1, 0, 1} or {0, 1, 2,3}. The input value is the activation value output after the operation of the previous layer in the convolutional neural network, and there are a large number of continuously stored values in the activation value of each layer. Assume that in the code stream corresponding to the input value, each quantized value and its consecutive number are represented by 8 bits. In an example, 8 bits are used to compress the activation value, and the first 6 bits are used for Count the number of consecutive values, and the last two bits are used to store the actual value. For an unsigned value with 2 quantization bits, 00 is used for 0, 01 for 1, 10 for 2, and 11 for 3 in binary. For a signed value with 2 quantization bits, 00 is used for 0, 01 for 1, 10 for -2, and 11 for -1 in binary. Please refer to Table 3. Taking the unsigned value as an example, the original value has 32 numbers, which are 6 consecutive 2s, 7 consecutive 1s, 7 consecutive 3s and 12 consecutive 0s. The binary representation of each number is 2 bits. , a total of 64 bits are required; the first sub-stream after encoding is "00011010", of which the first 6 bits "000110" represent the number of the last 2 bits "10"; the second sub-stream is " 00011101", the third sub-stream is "00011111", and the fourth sub-stream is "01010000", then the encoded stream is "000110100001110100011111101010000", which is expressed as 1A1D1F50 in hexadecimal format, and the encoding occupies 32 bits. Compared with the original value of 64 bits, the compression ratio is 50%, which is conducive to saving storage resources; when decoding, please refer to Table 4, the encoding is 1A1D1F50, that is, "00011010000111010001111101010000", which can be obtained according to the preset 8 bits To "00011010", "00011101", "00011111" and "01010000", the first 6 bits are used to count the number of consecutive values, and the last two bits are used to store the actual value. After decoding, they are 6 consecutive 2 , 7 consecutive 1s, 7 consecutive 3s and 12 consecutive 0s.

table 3

Table 4

In an embodiment, please refer to FIG. 4 , the processing device further includes an encoder 15, and the adder 13 is configured to transmit the operation result of the convolution operation to the encoder 15; the encoder 15. After the operation result of the convolution operation is encoded, the encoded result is stored in the memory, thereby helping to reduce the amount of data that needs to be stored and realize comprehensive utilization of storage resources. In an example, the operation result of the convolution operation may be encoded in a run-length encoding manner.

In one embodiment, referring to FIG. 5 , the processing device further includes a control module 16, which is configured to receive a convolution operation instruction and feed back the convolution operation instruction to the data loading module. 11. The decoder 14, the data selector 12 and the adder 13; the data loading module 11 loads the compressed input value and the compressed weight value according to the convolution operation instruction and inputs the In the decoder 14; the decoder 14 decompresses the compressed input value and the compressed weight value respectively according to the convolution operation instruction, and inputs it into the data selector 12; the data selector 12 based on the convolution operation instruction, and output the multiplication result in the convolution operation according to the input value and the weight value, and send the multiplication result to the adder 13; the adder 13 Based on the convolution operation instruction, and output the operation result of the convolution operation according to the multiplication operation result.

In one example, the control module may be a central processing unit (Central Processing Unit, CPU), or may be a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

Correspondingly, referring to FIG. 6 , an embodiment of the present application further provides a data processing method. The method is applied to a processing apparatus for performing convolution operations, and the processing apparatus includes a data selector; the method includes:

In step S101, the input value and the weight value are loaded.

In step S102, the input value and the weight value are input into the data selector, and the data selector is used to obtain the multiplication result in the convolution operation.

In step S103, an operation result of the convolution operation is obtained based on the multiplication operation result.

In this embodiment, the multiplication result in the convolution operation is obtained by using the data selector instead of the multiplier, which further reduces the complexity of the operation, thereby improving the operation speed of the neural network and meeting the real-time requirements of actual products.

In one embodiment, the input value and the weight value are quantized results.

In an embodiment, the number of quantization bits of the input value and/or the weight value is less than 8.

In this embodiment, when the convolution operation is performed, the quantized input value and the quantized weight value are used to perform the operation. Compared with before the quantization, less computing resources are consumed, and the required bandwidth is also reduced. power consumption.

In an embodiment, the inputting the input value and the weight value into a data selector, and using the data selector to obtain a multiplication result in the convolution operation includes: inverting the input value Operation to obtain the inverse of the input value; input the input value, the inverse of the input value and the weight value into a data selector, and use the data selector to obtain the multiplication operation in the convolution operation result.

In an embodiment, if the number of quantization bits of the weight value is 1, the obtaining the multiplication operation result in the convolution operation by using the data selector includes: according to the input weight value, making all The data selector selects to output the input value or the inverse of the input value.

In one embodiment, the input value and/or the inverse of the input value is represented in two's complement form.

In one embodiment, the data selector includes a gate circuit.

In one embodiment, the input value includes a signed value and an unsigned value; the input value input into the data selector is a signed value.

If the input value is an unsigned value, the obtaining the operation result of the convolution operation based on the multiplication operation result includes: obtaining the operation result of the convolution operation based on the multiplication operation result and the offset value, The offset value indicates a conversion relationship between signed and unsigned values.

In one embodiment, the offset value is determined according to the number of quantization bits of the input value and the weight value.

In one embodiment, the offset value is obtained offline.

In one embodiment, the obtaining the input value and the weight value includes: reading the compressed input value and the compressed weight value from a memory; Decompress to obtain the input value and weight value.

In this embodiment, the compressed input value and the compressed weight value are stored in the memory, which is beneficial to reduce the amount of data that needs to be stored and realize the comprehensive use of storage resources.

In an embodiment, the two code streams respectively indicating the compressed input value and the compressed weight value are obtained by encoding in a run-length encoding manner.

In one embodiment, if the number of quantization bits of the input value or the weight value is 1, the code streams corresponding to the compressed input value and the compressed weight value respectively indicate the number of consecutive quantization values. ; the quantized value is the quantized value of the input value and the weight value.

In one embodiment, if the number of quantization bits of the input value or the weight value is greater than 1, the code streams corresponding to the compressed input value or the compressed weight value respectively indicate the quantization value and the quantization value. The number of consecutive values; the quantized value is the quantized value of the input value or the weight value.

In one embodiment, if the number of quantization bits of the input value or the weight value is 1, in the code stream corresponding to the input value or the weight value respectively, the number of consecutive quantization values is pre-used. Set the number of bits to represent.

The separately decompressing the compressed input value and the compressed weight value includes: decompressing the compressed input value or the compressed weight value according to the preset number of bits, respectively. The corresponding code stream is divided to obtain one or more sub-code streams; according to the one or more sub-code streams and the quantization value corresponding to the sub-code stream, the decompressed input value or the decompressed weight value is obtained.

In an embodiment, the quantization values corresponding to the sub-streams are determined according to a preset correspondence relationship, and the correspondence indicates the quantization values corresponding to different orders of the sub-streams in the code stream.

In an embodiment, if the number of quantized bits of the input value or the weight value is greater than 1, in the code stream corresponding to the input value or the weight value, each quantization value and its consecutive number Use the preset number of bits to represent.

The decompressing the compressed input value and the compressed weight value respectively includes: corresponding to the compressed input value or the compressed weight value according to a preset number of bits, respectively; The code stream is divided to obtain one or more sub-code streams; for each sub-code stream, the quantized value is obtained from the specified bits of the sub-code stream and the consecutive number of quantized values are obtained from other bits. number.

The data processing device can be applied to common platforms such as ARM, DSP, and FPGA, and can be widely applied to UAVs, handheld devices, robots, and IoT devices.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium, such as a memory including instructions, executable by a processor of an apparatus to perform the above-described method. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the terminal, enable the terminal to execute the above method.

The embodiment of the present application provides a movable platform, including:

body;

a power system, arranged on the body, the power system is used to provide power for the movable platform;

A data processing device as described above. The data processing device can be used for tasks such as detection of objects and classification.

It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. The terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion such that a process, method, article or device comprising a list of elements includes not only those elements, but also other not expressly listed elements, or also include elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The methods and devices provided by the embodiments of the present application have been introduced in detail above, and specific examples are used to illustrate the principles and implementations of the present application. At the same time, for those of ordinary skill in the art, according to the idea of the application, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as a limitation to the application. .

Claims (36)

A data processing method, which is applied to a processing device for performing convolution operations, wherein the processing device includes a data selector; the method includes: Load input values and weight values; Inputting the input value and the weight value into the data selector, and using the data selector to obtain the multiplication result in the convolution operation; The operation result of the convolution operation is obtained based on the multiplication operation result. The method according to claim 1, wherein the input value and the weight value are quantized results. The method according to claim 2, wherein the number of quantization bits of the input value and/or the weight value is less than 8. The method according to claim 1, wherein the inputting the input value and the weight value into a data selector, and using the data selector to obtain a multiplication result in the convolution operation, comprising: Inverting the input value to obtain the inverse of the input value; The input value, the inverse of the input value, and the weight value are input into a data selector, and the data selector is used to obtain a multiplication result in the convolution operation. The method according to claim 4, wherein, if the number of quantization bits of the weight value is 1, the obtaining the multiplication operation result in the convolution operation by using the data selector comprises: According to the input weight value, the data selector is caused to select and output the input value or the opposite number of the input value. The method according to claim 1, wherein the input value and/or the inverse of the input value is represented in complement form. 6. The method of claim 5, wherein the data selector comprises a gate circuit. The method according to claim 1, wherein the input value includes a signed value and an unsigned value; the input value input into the data selector is a signed value; If the input value is an unsigned value, obtaining the operation result of the convolution operation based on the multiplication operation result, including: An operation result of the convolution operation is obtained based on the multiplication operation result and an offset value, the offset value indicating a conversion relationship between a signed value and an unsigned value. The method according to claim 8, wherein the offset value is determined according to the number of quantization bits of the input value and the weight value. The method according to claim 8 or 9, wherein the offset value is obtained offline. The method according to claim 1, wherein the acquiring the input value and the weight value comprises: reading compressed input values and compressed weight values from memory; The compressed input value and the compressed weight value are decompressed, respectively, to obtain the input value and the weight value. The method according to claim 11, wherein the two code streams indicating the compressed input value and the compressed weight value respectively are obtained through run-length coding. The method according to claim 12, wherein if the number of quantization bits of the input value or the weight value is 1, the code streams corresponding to the compressed input value and the compressed weight value respectively correspond to Indicates the number of consecutive quantized values; the quantized value is the quantized value of the input value and the weight value. The method according to claim 12, wherein if the number of quantization bits of the input value or the weight value is greater than 1, the code stream corresponding to the compressed input value or the compressed weight value respectively Indicates the quantized value and the number of consecutive quantized values; the quantized value is the quantized value of the input value or the weight value. The method according to claim 13, wherein if the number of quantization bits of the input value or the weight value is 1, in the code stream corresponding to the input value or the weight value, each quantization bit The number of consecutive values is represented by the preset number of bits; The decompressing the compressed input value and the compressed weight value, respectively, includes: Divide the code streams corresponding to the compressed input value or the compressed weight value according to the preset number of bits to obtain one or more sub-code streams; The decompressed input value or the decompressed weight value is obtained according to the one or more sub-streams and the quantization value corresponding to the sub-stream. The method according to claim 15, wherein the quantization value corresponding to the sub-code stream is determined according to a preset correspondence relationship, and the correspondence relationship indicates different orders of the sub-code stream in the code stream the corresponding quantization value. The method according to claim 14, wherein if the number of quantization bits of the input value or the weight value is greater than 1, in the code stream corresponding to the input value or the weight value, each quantization The value and its consecutive number are represented by the preset number of bits; The decompressing the compressed input value and the compressed weight value, respectively, includes: According to the preset number of bits, the code streams corresponding to the compressed input values or the compressed weight values are divided to obtain one or more sub-code streams; For each sub-stream, the quantized value is obtained from a specified bit of the sub-stream and the consecutive number of the quantized value is obtained from other bits. A data processing device, characterized in that the device is used to perform convolution operations, comprising a data loading module, a data selector and an adder; the data loading module, for loading the input value and the weight value and inputting it into the data selector; the data selector, configured to output the multiplication result in the convolution operation according to the input value and the weight value, and send the multiplication result to the adder; The adder is configured to output the operation result of the convolution operation according to the multiplication operation result. The apparatus according to claim 18, wherein the input value and the weight value are quantized results. The apparatus according to claim 19, wherein the number of quantization bits of the input value and/or the weight value is less than 8. The device according to claim 18, wherein the data selector is specifically configured to: output the multiplication operation in the convolution operation according to the input value, the inverse of the input value and the weight value result. The apparatus according to claim 21, wherein the number of quantization bits of the weight value is 1, and the data selector is specifically configured to: select and output the input value or the input value according to the input weight value The opposite of the value. The apparatus of claim 18, wherein the input value and/or the inverse of the input value is represented in two's complement form. 23. The apparatus of claim 22, wherein the data selector comprises a gate circuit. The apparatus according to claim 18, wherein the input value includes a signed value and an unsigned value; the input value input into the data selector is a signed value; If the input value is an unsigned value, the adder is specifically configured to: output the operation result of the convolution operation according to the multiplication operation result and the offset value; the offset value indicates the signed value and the unsigned value Conversion relationship between values. The apparatus according to claim 25, wherein the offset value is determined according to the number of quantization bits of the input value and the weight value. The apparatus of claim 25 or 26, wherein the offset value is obtained offline. The apparatus of claim 18, further comprising a decoder; The data loading module is specifically configured to read the compressed input value and the compressed weight value from the memory; The decoder is used to decompress the compressed input value and the compressed weight value, respectively, and input into the data selector. The apparatus according to claim 28, wherein the two code streams indicating the compressed input value and the compressed weight value respectively are obtained by encoding in a run-length encoding manner. The device according to claim 29, wherein if the number of quantization bits of the input value or the weight value is 1, the code streams corresponding to the compressed input value and the compressed weight value respectively correspond to Indicates the number of consecutive quantized values; the quantized value is the quantized value of the input value and the weight value. The device according to claim 29, wherein if the number of quantization bits of the input value or the weight value is greater than 1, the code stream corresponding to the compressed input value or the compressed weight value respectively Indicates the quantized value and the number of consecutive quantized values; the quantized value is the quantized value of the input value or the weight value. The apparatus according to claim 30, wherein if the number of quantization bits of the input value or the weight value is 1, in the code stream corresponding to the input value or the weight value, each quantization bit The number of consecutive values is represented by the preset number of bits; The decoder is specifically configured to: divide the code streams corresponding to the compressed input value or the compressed weight value respectively according to the preset number of bits to obtain one or more sub-code streams; The one or more sub-streams and the quantized values corresponding to the sub-streams are obtained as decompressed input values or decompressed weight values. The apparatus according to claim 32, wherein the quantization values corresponding to the sub-streams are determined according to a preset correspondence relationship, and the correspondence indicates different orders of the sub-streams in the code stream the corresponding quantization value. The apparatus according to claim 31, wherein if the number of quantization bits of the input value or the weight value is greater than 1, in the code stream corresponding to the input value or the weight value, each quantization The value and its consecutive number are represented by the preset number of bits; The decoder is specifically configured to: divide the compressed input value or the code stream corresponding to the compressed weight value according to a preset number of bits to obtain one or more sub-code streams; for each A sub-stream, the quantized value is obtained from a specified bit of the sub-stream and the consecutive number of the quantized value is obtained from other bits. A computer-readable storage medium, characterized in that computer instructions are stored thereon, and when the instructions are executed by a processor, perform the method of any one of claims 1 to 17. A movable platform, characterized in that, comprising: body; a power system, arranged on the body, the power system is used to provide power for the movable platform; A data processing device as claimed in claims 18-33.

Images