CN111027683A - Data processing method, data processing device, storage medium and electronic equipment - Google Patents
Data processing method, data processing device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN111027683A CN111027683A CN201911253883.6A CN201911253883A CN111027683A CN 111027683 A CN111027683 A CN 111027683A CN 201911253883 A CN201911253883 A CN 201911253883A CN 111027683 A CN111027683 A CN 111027683A
- Authority
- CN
- China
- Prior art keywords
- data
- pooling
- intermediate data
- calculation
- outside
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Complex Calculations (AREA)
Abstract
本申请实施例提供一种数据处理方法、装置、存储介质及电子设备，数据处理方法应用于神经网络处理器，所述方法包括：从外部获取数据；将所述数据进行多次卷积计算，得到多个第一中间数据；对每个所述第一中间数据进行池化计算，得到多个第二中间数据，并将多个所述第二中间数据进行计算得到目标数据；以及将所述目标数据向外部发送。卷积计算和池化计算都在神经网络处理器中进行，而且可以进行多次池化计算后再将计算后的目标数据向外部发送，优化了池化计算，减小了神经网络处理器和外部模块的数据交互，减小了数据吞吐量，提高了池化计算的效率，提高了神经网络整体计算效率。
Embodiments of the present application provide a data processing method, device, storage medium and electronic device. The data processing method is applied to a neural network processor, and the method includes: acquiring data from outside; performing multiple convolution calculations on the data, obtaining a plurality of first intermediate data; performing pooling calculation on each of the first intermediate data to obtain a plurality of second intermediate data, and calculating a plurality of the second intermediate data to obtain target data; and The target data is sent to the outside. Both convolution calculation and pooling calculation are performed in the neural network processor, and multiple pooling calculations can be performed before sending the calculated target data to the outside, which optimizes the pooling calculation and reduces the number of neural network processors and The data interaction of external modules reduces the data throughput, improves the efficiency of pooled computing, and improves the overall computing efficiency of the neural network.
Description
Technical Field
The present disclosure relates to the field of electronic technologies, and in particular, to a data processing method and apparatus, a storage medium, and an electronic device.
Background
A neural-Network Processing Unit (NPU) is a type of processing chip for accelerating the operation of a neural network. Convolution calculation and Pooling calculation are used in the neural network, and Pooling (Pooling) is a commonly used operator in the neural network. In the calculation process of the neural network, data is frequently exchanged between the NPU and the external processor, so that a large amount of data throughput is brought, and the overall calculation efficiency of the neural network is not high.
Disclosure of Invention
The embodiment of the application provides a data processing method, a data processing device, a storage medium and electronic equipment, which can improve the overall calculation efficiency of a neural network.
In a first aspect, an embodiment of the present application provides a data processing method, which is applied to a neural network processor, and the method includes:
acquiring data from the outside;
performing convolution calculation on the data for multiple times to obtain multiple first intermediate data;
performing pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and calculating the plurality of second intermediate data to obtain target data; and
and sending the target data to the outside.
In a second aspect, an embodiment of the present application further provides a data processing apparatus, which is applied to a neural network processor, and the apparatus includes:
the data acquisition module is used for acquiring data from the outside;
the first intermediate data acquisition module is used for carrying out convolution calculation on the data for multiple times to obtain a plurality of first intermediate data;
the target data acquisition module is used for performing pooling calculation on each first intermediate data to obtain a plurality of second intermediate data and calculating the plurality of second intermediate data to obtain target data; and
and the sending module is used for sending the target data to the outside.
In a third aspect, embodiments of the present application further provide a storage medium, on which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the data processing method as described above.
In a fourth aspect, an embodiment of the present application further provides an electronic device, which includes a neural network processor and a memory, where the memory has a computer program, and the neural network processor is configured to execute the data processing method described above by calling the computer program.
In the embodiment of the application, convolution calculation and pooling calculation are carried out in the neural network processor, and calculated target data can be sent to the outside after being subjected to multiple times of pooling calculation, so that pooling calculation is optimized, data do not need to be acquired from the outside every time of pooling calculation, and the data are not needed to be sent to the outside every time of pooling calculation, so that data interaction between the neural network processor and an external module is reduced, data throughput is reduced, the efficiency of pooling calculation is improved, and the overall calculation efficiency of the neural network is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below.
Fig. 1 is a schematic flowchart of a first data processing method according to an embodiment of the present application.
Fig. 2 is a schematic flowchart of a second data processing method according to an embodiment of the present application.
Fig. 3 is a third flowchart illustrating a data processing method according to an embodiment of the present application.
Fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
Fig. 5 is a schematic structural diagram of a first electronic device according to an embodiment of the present application.
Fig. 6 is a schematic structural diagram of a second electronic device according to an embodiment of the present application.
Detailed Description
The following description is based on illustrated embodiments of the application and should not be taken as limiting the application with respect to other embodiments that are not detailed herein. The term "module" as used herein may be considered a software object executing on the computing system. The various modules, engines, and services herein may be considered as objects of implementation on the computing system.
An execution main body of the data processing method can be the data processing device provided by the embodiment of the application or an electronic device integrated with the data processing device. The electronic device may be a smart phone, a tablet computer, a Personal Digital Assistant (PDA), or the like, or may be a game device, an AR (Augmented Reality) device, a vehicle-mounted device, a data storage device, an audio playing device, a video playing device, a notebook computer, a desktop computing device, or the like.
The following is a detailed description of the analysis.
An embodiment of the present application provides a data processing method, which is applied to a neural-Network Processing Unit (NPU), please refer to fig. 1, where fig. 1 is a first flowchart of the data processing method provided in the embodiment of the present application, and the data processing method may include the following steps:
101, data is acquired from the outside.
The NPU acquires data to be calculated from the outside. The data to be calculated may be stored in a memory outside the NPU, and the NPU may obtain the data to be calculated through a data bus, an external processor, or the like.
And 102, carrying out convolution calculation on the data for multiple times to obtain multiple first intermediate data.
After the NPU acquires data from the outside, the NPU may perform convolution calculation on the data for a plurality of times, thereby obtaining a plurality of first intermediate data. The convolution calculation in the neural network occupies about 80% of the calculation amount, the calculation pressure of a Central Processing Unit (CPU) can be reduced by performing the convolution calculation through the NPU, and meanwhile, the neural network calculation can be performed more quickly by using a special NPU.
103, performing pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and calculating the plurality of second intermediate data to obtain target data.
The NPU may also perform a pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and perform a calculation using the plurality of second intermediate data to obtain the target data.
It should be noted that obtaining target data through a plurality of pooling calculations may be understood as a total pooling operation. The amount of data required for multiple pooling calculations (i.e., one total pooling operation) is related to the size of the kernel (kernel) of the pooling calculation. Therefore, before performing the pooling calculation, it is necessary to arrange the size information (length, width, depth) of the core of the pooling calculation in the pooling calculation unit, and the plurality of second intermediate data can also be understood as data satisfying the size of the core of the pooling calculation.
The number of convolution calculations performed by the NPU is equal to the number of kernels of the pooling calculation, that is, the number of convolution calculations performed by the NPU is equal to the size of the kernels of the pooling calculation.
And 104, transmitting the target data to the outside.
The NPU performs multiple convolution calculations and multiple pooling calculations on data needing to be calculated externally to obtain target data, and then sends the target data to the outside. In the process of multiple convolution calculations and multiple pooling calculations, the NPU can perform multiple pooling calculations once, namely, the NPU performs data interaction with the outside after performing multiple pooling calculations, the NPU has less data interaction with the outside (such as an external storage module or an external processor), the NPU has less overall data throughput with the outside, the efficiency of pooling calculations can be improved, and the overall calculation efficiency of the neural network is improved.
The method described in the previous embodiment is further detailed by way of example.
Referring to fig. 2, fig. 2 is a second flowchart illustrating a data processing method according to an embodiment of the present disclosure. The data processing method may specifically include:
data is acquired from the outside 201.
The NPU acquires data to be calculated from the outside. The data to be calculated may be stored in a memory outside the NPU, and the NPU may obtain the data to be calculated through a data bus, an external processor, or the like.
And 202, carrying out convolution calculation on the data for multiple times to obtain multiple first intermediate data.
After the NPU acquires data from the outside, the NPU may perform convolution calculation on the data for a plurality of times, thereby obtaining a plurality of first intermediate data. The convolution calculation in the neural network occupies about 80% of the calculation amount, the calculation pressure of a Central Processing Unit (CPU) can be reduced by performing the convolution calculation through the NPU, and meanwhile, the neural network calculation can be performed more quickly by using a special NPU.
The NPU may perform convolution calculation on all externally acquired data, and obtain a plurality of first intermediate data after all convolution calculations are completed.
And 203, storing the plurality of first intermediate data to the outside.
The first intermediate data may be stored outside the NPU after it is obtained. Such as caches, memory, etc. outside the NPU.
And 204, acquiring a plurality of first intermediate data from the outside.
The NPU then retrieves the first plurality of intermediate data after convolution calculation from an external source (e.g., a cache external to the NPU).
205, performing pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and calculating the plurality of second intermediate data to obtain target data.
The NPU may also perform a pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and perform a calculation using the plurality of second intermediate data to obtain the target data. The first intermediate data may be stored in a storage space in the NPU, and then pooled calculation is performed on each first intermediate data, or one or more first intermediate data may be obtained from the outside, and after the pooled calculation is completed, one or more first intermediate data may be obtained from the outside until the required first intermediate data is obtained. The data of the convolution calculation is not directly sent to the pooling calculation unit, but is stored in an external storage, and the data of the convolution calculation is obtained from the external storage before the pooling calculation.
It should be noted that obtaining target data through a plurality of pooling calculations may be understood as a total pooling operation. The amount of data required for multiple pooling calculations (i.e., one total pooling operation) is related to the size of the kernel (kernel) of the pooling calculation. Therefore, before performing the pooling calculation, it is necessary to arrange the size information (length, width, depth) of the core of the pooling calculation in the pooling calculation unit, and the plurality of second intermediate data can also be understood as data satisfying the size of the core of the pooling calculation.
In some embodiments, the number of convolution calculations performed by the NPU is equal to the number of kernels of the pooling calculation, i.e. the number of convolution calculations performed by the NPU is equal to the size of the kernels of the pooling calculation.
In some embodiments, the NPU may perform a very large number of convolution calculations, resulting in a very large number of first intermediate data. In the process of performing pooling calculation by the NPU, pooling calculation may be performed according to the size of the pooling core, and specifically, one or more first intermediate data may be obtained each time, pooling calculation may be performed on the one or more first intermediate data to obtain one second intermediate data or a plurality of second intermediate data, pooling calculation may be performed on the one or more first intermediate data after pooling calculation is completed to obtain one or more second intermediate data, and target data formed by calculating the plurality of second intermediate data may be obtained until the pooling calculation times or the second intermediate data reaches the size of the pooling core.
The target data may be obtained in various ways, including, for example, a maximum pooling way, a minimum pooling way, and an average pooling way. The maximum value pooling mode is to compare data of each point in the second intermediate data to obtain a maximum value of each point and form target data. The minimum pooling mode is to compare the data of each point in the second intermediate data to obtain the minimum value of each point and form target data. The average value pooling mode is to accumulate the data of each point in the second intermediate data; and averaging the accumulated data of each point to form target data. For ease of understanding, various ways of obtaining the target data are described below.
The maximum pooling mode specifically comprises: after the first second intermediate data is obtained, the second intermediate data is stored; after second intermediate data is obtained, comparing the data of each point in the first second intermediate data and the second intermediate data to obtain maximum data on each point, forming second intermediate data after comparison, after third second intermediate data is obtained, comparing the third second intermediate data with the second intermediate data after comparison to obtain maximum data on each point, forming new second intermediate data after comparison, and so on until all the second intermediate data are compared to obtain target data of which each point is the maximum data.
The minimum pooling mode specifically comprises: after the first second intermediate data is obtained, the second intermediate data is stored; and after second intermediate data is obtained, comparing the data of each point in the first second intermediate data and the second intermediate data to obtain the minimum data on each point, forming the second intermediate data after comparison, after third second intermediate data is obtained, comparing the third second intermediate data with the second intermediate data after comparison to obtain the minimum data on each point, forming new second intermediate data after comparison, and so on until all the second intermediate data are compared to obtain target data of which each point is the minimum data.
The average value pooling mode specifically comprises the following steps: after the first second intermediate data is obtained, the second intermediate data is stored; after second intermediate data is obtained, point-to-point data accumulation is carried out on the first second intermediate data and the second intermediate data to form accumulated second intermediate data, point-to-point data accumulation is carried out on the third second intermediate data and the accumulated second intermediate data to form new accumulated second intermediate data, and the like is carried out until all second intermediate data are accumulated, and then division (dividing by the number of the second intermediate data) is carried out to obtain target data of which each point is an average value. It should be noted that division preserves integer bits by rounding.
It should be noted that the target data may be obtained in one or two or three of a maximum pooling manner, a minimum pooling manner and an average pooling manner, and it may also be understood that the target data may include one or two or three of a maximum pooling data, a minimum pooling data and an average pooling data, which is convenient for the convolutional network to call.
And 206, transmitting the target data to the outside.
And the NPU performs multi-time pooling calculation to obtain target data, and then sends the target data to the outside. In the process of multiple pooling calculations, the NPU can perform multiple pooling calculations once, namely the NPU performs data interaction with the outside after performing multiple pooling calculations, the NPU has less data interaction with the outside (such as an external storage module or an external processor), the NPU has less overall data throughput with the outside, the efficiency of the pooling calculations can be improved, and the overall calculation efficiency of the neural network is improved.
The embodiment of the present application further provides a data processing method, and specifically please refer to fig. 3, where fig. 3 is a third schematic flow chart of the data processing method provided in the embodiment of the present application. The data processing method may specifically include:
301, data is acquired from the outside.
The NPU acquires data to be calculated from the outside. The data to be calculated may be stored in a memory outside the NPU, and the NPU may obtain the data to be calculated through a data bus, an external processor, or the like.
And 302, performing convolution calculation on the data to obtain first intermediate data.
After the NPU acquires data from the outside, the NPU may perform convolution calculation on the data once, thereby obtaining a first intermediate data.
303, performing pooling calculation on the first intermediate data to obtain second intermediate data.
The NPU may also perform pooling calculations on the first intermediate data obtained by the convolution calculations to obtain a second intermediate data.
The data of the pooling calculation is directly sent to the pooling calculation unit after the convolution calculation.
And 304, judging whether the pooling calculation reaches the pooling calculation core number.
It should be noted that one total pooling operation is related to the number of cores (core size) of the pooling calculation. Therefore, before pooling calculation, it is necessary to arrange the size information (length, width, depth) of the cores of the pooling calculation to the pooling calculation unit. Whether the pooling calculation reaches the pooling calculation core number or not can be judged, and whether a total pooling operation is reached or not can be understood.
305, if the pooling calculation does not reach the pooling calculation core number, skipping to the corresponding step 302.
If the pooling calculation does not reach the pooling calculation core number, the pooling calculation can be continued.
And 306, if the pooling calculation reaches the pooling calculation core number, calculating the second intermediate data obtained according to the pooling calculation to obtain target data.
If the pooling calculation reaches the pooling calculation core number, the pooling calculation number is already reached, and the pooling calculation cannot be performed any more, and at this time, the target data can be calculated according to the second intermediate data obtained by the pooling calculation.
There are various ways to obtain the target data, including, for example, a maximum pooling way, a minimum pooling way, and an average pooling way. The maximum pooling mode, the minimum pooling mode and the average pooling mode may be specifically defined in the above embodiments, and are not described herein again.
For ease of understanding, the manner in which the target data is obtained will be described below.
Illustratively, an NPU Convolution calculation (Convolution) may output 1x1x64(HWC) output data.
The amount of data required for one Pooling calculation (Pooling) is related to the Pooling core (kernel) size. Therefore, before Pooling, the kernel size information (length, width, depth) of Pooling needs to be configured to the Pooling cell. Convolition outputs 1x1x64(HWC) data at a time, so the Pooling unit calculates 1x1x64(HWC) data at a time until the previously configured Pooling size is reached.
Assuming that kernel of Pooling is 3x3x64(HWC), Max Pooling (Max Pooling) is selected, then first Convolution outputs 1x1x64(HWC) to Pooling cell each time, and the Pooling cell holds the 1x1x64 number; and outputting 1x1x64(HWC) to the Pooling unit for the second time, sequentially comparing the 64 numbers by the Pooling unit to obtain the maximum value on each point, and repeating the steps until the Pooling unit obtains the 9 th Convolition output.
Minimum Pooling (Min Pooling) is similar to Max Pooling, comparing 64 numbers at a time, yielding a minimum at each point.
The average Pooling (Avg Pooling) is slightly different from the maximum Pooling, and the 64 accumulated values are accumulated point-to-point each time in the average Pooling to obtain 64 accumulated values, and division (division by 9) is performed again until the 9 th output is obtained by the Pooling unit to obtain 64 average values. It should be noted that division preserves integer bits by rounding.
307, the target data is transmitted to the outside.
And the NPU performs multi-time pooling calculation to obtain target data, and then sends the target data to the outside. In the process of multiple pooling calculations, the NPU can perform multiple pooling calculations once, namely the NPU performs data interaction with the outside after performing multiple pooling calculations, the NPU has less data interaction with the outside (such as an external storage module or an external processor), the NPU has less overall data throughput with the outside, the efficiency of the pooling calculations can be improved, and the overall calculation efficiency of the neural network is improved.
As can be seen from the above, in the data processing method provided in this embodiment, the NPU performs multiple convolution calculations and multiple pooling calculations on data to be calculated externally to obtain target data, and then sends the target data to the outside. In the process of multiple convolution calculations and multiple pooling calculations, the NPU can perform multiple pooling calculations once, namely, the NPU performs data interaction with the outside after performing multiple pooling calculations, the NPU has less data interaction with the outside (such as an external storage module or an external processor), the NPU has less overall data throughput with the outside, the efficiency of pooling calculations can be improved, and the overall calculation efficiency of the neural network is improved.
It can be understood that, in the above embodiment, the first intermediate data obtained by the convolution calculation may be temporarily stored in the first buffer storage unit, and when the pooling calculation is required, the first intermediate data is obtained from the first buffer storage unit to perform the pooling calculation. Without storing in an external memory, such as a memory outside the NPU. Specifically, when the first intermediate data stored in the first buffer unit reaches a first threshold, the first intermediate data stored in the first buffer unit is pooled.
When the first intermediate data stored in the first buffer unit reaches the first threshold, the first intermediate data stored in the first buffer unit may be pooled to release the storage space of the first buffer unit. For example, when the first intermediate data stored in the first buffer unit reaches the first threshold, the first intermediate data may be read from the first buffer unit, and the first intermediate data may be pooled to obtain the second intermediate data. Or when the first intermediate data stored in the first buffer unit reaches the first threshold, the first buffer unit may directly output the first intermediate data to a unit where the NPU performs pooling calculation.
In the process of performing pooling processing on the first intermediate data, the size of the pooling window and the result output address may be obtained first, and the pooling processing may be performed on the first intermediate data according to the pooling window to obtain second intermediate data. For example, the central processing unit may generate a configuration instruction when starting the pooling computing unit of the NPU, where the configuration instruction carries configuration parameters such as the size of the pooling window and the result output address, and the NPU may perform pooling processing on the first intermediate data according to the configuration parameters. Configuration parameters such as the size of the pooling window and the result output address may also be stored in an external storage device, and the NPU may acquire the configuration parameters from the external storage device before pooling calculation.
In some other embodiments, the neural network processor may further include a second buffer unit for storing the second intermediate data, and may further store the first intermediate data together with the first buffer unit. The data processing method may further include:
when the first intermediate data stored in the first buffer register unit reaches a first threshold value, pooling processing is performed on the first intermediate data stored in the first buffer register unit.
When the first intermediate data stored in the first buffer register unit reaches a first threshold value and the second intermediate data stored in the second buffer register unit does not reach a second threshold value, new first intermediate data is stored in the second buffer register unit.
When the first intermediate data stored in the first buffer register unit does not reach the first threshold value and the second intermediate data stored in the second buffer register unit reaches the second threshold value, the new first intermediate data is stored in the first buffer register unit.
When the storage space of the first buffer register unit is full, the neural network processor performs convolution processing again to obtain new first intermediate data without storage space, and at this time, the neural network processor can store the new first intermediate data obtained by performing convolution processing on the data to be processed into the second buffer register unit of which the storage space is not full. When the new first intermediate data stored in the second buffer unit reaches the second threshold, the neural network processor may perform pooling processing on the new first intermediate data stored in the second buffer unit to release the storage space of the second buffer unit. Because the storage space of the first buffer unit has been gradually released, the neural network processor may store the new first intermediate data obtained by the continuous calculation into the first buffer unit when the new first intermediate data stored in the second buffer unit reaches the second threshold, and circulate the process until all the data to be processed are processed.
The first threshold and the second threshold may be set identically or differently, for example, the first threshold and the second threshold may be set differently, so as to stagger the time of releasing the storage space of the two buffer units. The first threshold may be equal to a maximum data size of the first intermediate data that can be stored in the first buffer unit, so as to fully utilize the storage space of the first buffer unit. The second threshold may be equal to a maximum data amount of the first intermediate data that can be stored by the second buffer unit, so as to fully utilize the storage space of the second buffer unit. It can be understood that, if the first threshold and the second threshold are set to be values capable of storing the maximum data amount, when the storage space of the first buffer unit is full and the storage space of the second buffer unit is not full, the first intermediate data is stored in the second buffer unit; and when the storage space of the second buffer register unit is full and the storage space of the first buffer register unit is not full, storing the first intermediate data into the first buffer register unit. According to the embodiment of the application, the operation process of the neural network processor can be relatively continuous through the alternate storage of the two buffer register units, and the processing rate of the neural network processor is further improved.
Of course, the first threshold may be set to be smaller than the maximum data amount of the first intermediate data that can be stored in the first buffer unit, and the second threshold may be set to be smaller than the maximum data amount of the first intermediate data that can be stored in the second buffer unit.
In order to better implement the above method, an embodiment of the present application further provides a data processing apparatus, which is applied to a neural network processor, and specifically, refer to fig. 4, where fig. 4 is a schematic structural diagram of the data processing apparatus provided in the embodiment of the present application. The data processing apparatus 400 may include a data acquisition module 410, a first intermediate data acquisition module 420, a target data acquisition module 430, and a transmission module 440.
The data acquisition module 410 is used for acquiring data from the outside;
the first intermediate data obtaining module 420 is configured to perform convolution calculation on the data for multiple times to obtain multiple first intermediate data;
the target data obtaining module 430 is configured to perform pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and calculate the plurality of second intermediate data to obtain target data; and a sending module 440 for sending the target data to the outside.
In some embodiments, in calculating the plurality of second intermediate data to obtain the target data, the target data obtaining module 430 is further configured to compare data of each point in the plurality of second intermediate data to obtain a maximum value or a minimum value at each point, and form the target data.
In some embodiments, in calculating the plurality of second intermediate data to obtain the target data, the target data obtaining module 430 is further configured to accumulate data of each point in the plurality of second intermediate data; and averaging the accumulated data of each point to form target data.
In some embodiments, in performing the pooling calculation for each first intermediate data, the target data obtaining module 430 is further configured to perform a pooling calculation after each convolution calculation to obtain the first intermediate data.
In some embodiments, after performing pooling calculation once after obtaining the first intermediate data by convolution calculation, the target data obtaining module 430 is further configured to calculate a plurality of second intermediate data to obtain the target data if the pooling calculation reaches the preset pooling kernel number.
In some embodiments, before performing the pooling calculation on each first intermediate data, the first intermediate data obtaining module 420 is further configured to store the plurality of first intermediate data to the outside; and acquiring a plurality of first intermediate data from the outside.
In some embodiments, the number of convolution calculations is equal to the number of kernels of the pooling calculation.
The embodiment of the application also provides a neural network processor, and the neural network processor comprises the data processing device. Or the data processing apparatus may perform the above-described data processing method.
Referring to fig. 5, an electronic device 500 includes a neural network processor 501 and a memory 502. The neural network processor 501 is electrically connected to the memory 502.
The memory 502 may be used to store software programs and data, and the neural network processor 501 executes various functional applications and data processing by running computer programs stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store a computer program or the like required for at least one function; the storage data area may store data or the like according to the needs of the neural network processor 501.
Further, the memory 502 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 502 may also include a memory controller to provide the neural network processor 501 access to the memory 502.
In this embodiment of the application, the neural network processor 501 in the electronic device 500 loads instructions corresponding to one or more computer program processes into the memory 502, and the neural network processor 501 runs the computer program stored in the memory 502, so as to implement various functions as follows:
acquiring data from the outside;
carrying out convolution calculation on the data for multiple times to obtain multiple first intermediate data;
performing pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and calculating the plurality of second intermediate data to obtain target data;
and transmitting the target data to the outside.
Referring to fig. 6, fig. 6 is a second schematic structural diagram of an electronic device according to an embodiment of the present disclosure, where the electronic device 600 includes: a main processor 601, a memory 602, a camera assembly 603, a display 604, an audio circuit 605, a radio frequency circuit 606, a power supply 607, and a neural network processor 608. The camera module 603, the display 604, the audio circuit 605, the rf circuit 606, the power supply 607 and the neural network processor 608 are electrically connected to the main processor 601, respectively.
The main processor 601 may be a Central Processing Unit (CPU), which is a control center of the electronic device 600 and connects various parts of the electronic device 600 by using various interfaces and lines.
The memory 602 may be used to store software programs and modules, and the main processor 601 executes various functional applications and data processing by executing computer programs and data stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the electronic device, and the like. The main processor 601 may perform various functions of the electronic device 600 and process data by running or loading computer programs stored in the memory 602 and calling data stored in the memory 602.
The camera assembly 603 may include Image Processing circuitry, which may be implemented using hardware and/or software components, and may include various Processing units that define an Image Signal Processing (Image Signal Processing) pipeline. The image processing circuit may include at least: a plurality of cameras, an Image Signal Processor (ISP), a control logic, and an Image memory. Where each camera may include at least one or more lenses and an image sensor. The image sensor may include an array of color filters (e.g., Bayer filters). The image sensor may acquire light intensity and wavelength information captured with each imaging pixel of the image sensor and provide a set of raw image data that may be processed by an image signal processor.
The display 604 may be used to display information entered by or provided to the user as well as various graphical user interfaces, which may be comprised of graphics, text, icons, video, and any combination thereof.
The audio circuit 605 may be used to provide an audio interface between the user and the electronic device through a speaker, microphone.
The rf circuit 606 may be used for transceiving rf signals to establish wireless communication with a network device or other electronic devices through wireless communication, and for transceiving signals with the network device or other electronic devices.
The power supply 607 may be used to power various components of the electronic device 600. In some embodiments, the power supply 607 may be logically connected to the main processor 601 through a power management system, so as to implement functions of managing charging, discharging, and power consumption management through the power management system.
The neural network processor 608 may be used to perform neural network computations. Which may collectively perform neural network computations through data interaction with the host processor 601 and/or memory 602.
In the embodiment of the present application, the neural network processor 608 in the electronic device 600 loads instructions corresponding to one or more computer program processes into the memory 602, and the neural network processor 608 runs the computer program stored in the memory 602, so as to implement various functions as follows:
acquiring data from the outside;
carrying out convolution calculation on the data for multiple times to obtain multiple first intermediate data;
performing pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and calculating the plurality of second intermediate data to obtain target data;
and transmitting the target data to the outside.
In some embodiments, when calculating the plurality of second intermediate data to obtain the target data, the neural network processor 608 may perform:
and comparing the data of each point in the plurality of second intermediate data to obtain the maximum value or the minimum value of each point, and forming target data.
In some embodiments, when calculating the plurality of second intermediate data to obtain the target data, the neural network processor 608 may perform:
accumulating the data of each point in the second intermediate data;
and averaging the accumulated data of each point to form target data.
In some embodiments, in performing the pooling computation for each first intermediate datum, the neural network processor 608 may perform:
and performing pooling calculation once after the first intermediate data is obtained by convolution calculation each time.
In some embodiments, after performing the pooling calculation each time the convolution calculation obtains the first intermediate data, the neural network processor 608 may perform:
and if the pooling calculation reaches the preset pooling kernel number, calculating the plurality of second intermediate data to obtain target data.
In some embodiments, prior to performing the pooling calculation for each first intermediate data, the neural network processor 608 may perform:
storing the plurality of first intermediate data to the outside;
a plurality of first intermediate data are acquired from the outside.
In some embodiments, the number of convolution calculations is equal to the number of kernels of the pooling calculation.
An embodiment of the present application further provides a storage medium, where the storage medium stores a computer program, and when the computer program runs on a computer, the computer is caused to execute the data processing method in any one of the above embodiments, for example: acquiring data from the outside; carrying out convolution calculation on the data for multiple times to obtain multiple first intermediate data; performing pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and calculating the plurality of second intermediate data to obtain target data; and transmitting the target data to the outside.
In the embodiment of the present application, the storage medium may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
It should be noted that, for the data processing method of the embodiment of the present application, it can be understood by a person skilled in the art that all or part of the process of implementing the data processing method of the embodiment of the present application can be completed by controlling the relevant hardware through a computer program, where the computer program can be stored in a computer readable storage medium, such as a memory of an electronic device, and executed by a neural network processor in the electronic device, and during the execution process, the process of the embodiment of the data processing method can be included as the process of the embodiment of the data processing method. The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, etc.
For the data processing device of the embodiment of the present application, the functional modules thereof may be integrated into one neural network processor. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented as a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium such as a read-only memory, a magnetic or optical disk, or the like.
In the description of the present application, it is to be understood that terms such as "first", "second", and the like are used merely to distinguish one similar element from another, and are not to be construed as indicating or implying relative importance or implying any indication of the number of technical features indicated.
The data processing method, the data processing apparatus, the storage medium, and the electronic device provided in the embodiments of the present application are described in detail above. The principles and implementations of the present application are described herein using specific examples, which are presented only to aid in understanding the present application. Meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A data processing method is applied to a neural network processor, and is characterized by comprising the following steps:
acquiring data from the outside;
performing convolution calculation on the data for multiple times to obtain multiple first intermediate data;
performing pooling calculation on each first intermediate data to obtain a plurality of second intermediate data, and calculating the plurality of second intermediate data to obtain target data; and
and sending the target data to the outside.
2. The data processing method according to claim 1, wherein the calculating the plurality of second intermediate data to obtain the target data comprises:
and comparing the data of each point in the second intermediate data to obtain the maximum value or the minimum value of each point and form target data.
3. The data processing method according to claim 1, wherein the calculating the plurality of second intermediate data to obtain the target data comprises:
accumulating the data of each point in the second intermediate data; and
and averaging the accumulated data of each point to form target data.
4. A data processing method according to any of claims 1 to 3, wherein said performing a pooling calculation for each of said first intermediate data comprises:
and performing pooling calculation once after the first intermediate data is obtained by convolution calculation each time.
5. The data processing method of claim 4, wherein after performing a pooling calculation after each convolution calculation to obtain the first intermediate data, further comprising:
and if the pooling calculation reaches the preset pooling kernel number, calculating the plurality of second intermediate data to obtain target data.
6. A data processing method according to any one of claims 1 to 3, wherein before performing the pooling calculation for each of the first intermediate data, further comprising:
storing a plurality of the first intermediate data to the outside; and
a plurality of the first intermediate data is acquired from the outside.
7. The data processing method of claim 1, wherein the number of convolution calculations is equal to the number of kernels of pooling calculations.
8. A data processing device applied to a neural network processor is characterized by comprising:
the data acquisition module is used for acquiring data from the outside;
the first intermediate data acquisition module is used for carrying out convolution calculation on the data for multiple times to obtain a plurality of first intermediate data;
the target data acquisition module is used for performing pooling calculation on each first intermediate data to obtain a plurality of second intermediate data and calculating the plurality of second intermediate data to obtain target data; and
and the sending module is used for sending the target data to the outside.
9. A storage medium having stored thereon a computer program, characterized in that, when the computer program runs on a computer, it causes the computer to execute a data processing method according to any one of claims 1 to 7.
10. An electronic device comprising a neural network processor and a memory, the memory having a computer program, wherein the neural network processor is configured to perform the data processing method of any one of claims 1 to 7 by invoking the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911253883.6A CN111027683A (en) | 2019-12-09 | 2019-12-09 | Data processing method, data processing device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911253883.6A CN111027683A (en) | 2019-12-09 | 2019-12-09 | Data processing method, data processing device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111027683A true CN111027683A (en) | 2020-04-17 |
Family
ID=70208336
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911253883.6A Pending CN111027683A (en) | 2019-12-09 | 2019-12-09 | Data processing method, data processing device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111027683A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116531764A (en) * | 2022-01-25 | 2023-08-04 | 腾讯科技（深圳）有限公司 | Game data processing method, device, equipment and computer readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9721203B1 (en) * | 2016-11-10 | 2017-08-01 | Google Inc. | Performing kernel striding in hardware |
| US20180365520A1 (en) * | 2017-06-14 | 2018-12-20 | SK Hynix Inc. | Convolution neural network and a neural network system having the same |
| CN109615071A (en) * | 2018-12-25 | 2019-04-12 | 济南浪潮高新科技投资发展有限公司 | An energy-efficient neural network processor, acceleration system and method |
| CN109740732A (en) * | 2018-12-27 | 2019-05-10 | 深圳云天励飞技术有限公司 | Neural network processor, convolutional neural network data multiplexing method and related equipment |
| US20190236440A1 (en) * | 2018-01-31 | 2019-08-01 | Pin-Han Ho | Deep convolutional neural network architecture and system and method for building the deep convolutional neural network architecture |
-
2019
- 2019-12-09 CN CN201911253883.6A patent/CN111027683A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9721203B1 (en) * | 2016-11-10 | 2017-08-01 | Google Inc. | Performing kernel striding in hardware |
| US20180365520A1 (en) * | 2017-06-14 | 2018-12-20 | SK Hynix Inc. | Convolution neural network and a neural network system having the same |
| US20190236440A1 (en) * | 2018-01-31 | 2019-08-01 | Pin-Han Ho | Deep convolutional neural network architecture and system and method for building the deep convolutional neural network architecture |
| CN109615071A (en) * | 2018-12-25 | 2019-04-12 | 济南浪潮高新科技投资发展有限公司 | An energy-efficient neural network processor, acceleration system and method |
| CN109740732A (en) * | 2018-12-27 | 2019-05-10 | 深圳云天励飞技术有限公司 | Neural network processor, convolutional neural network data multiplexing method and related equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116531764A (en) * | 2022-01-25 | 2023-08-04 | 腾讯科技（深圳）有限公司 | Game data processing method, device, equipment and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3798850B1 (en) | On-chip code breakpoint debugging method, on-chip processor, and chip breakpoint debugging system | |
| US11449242B2 (en) | Shared storage space access method, device and system and storage medium | |
| CN110660066A (en) | Network training method, image processing method, network, terminal equipment and medium | |
| CN110929860B (en) | Convolution acceleration operation method and device, storage medium and terminal equipment | |
| CN110796624B (en) | Image generation method and device and electronic equipment | |
| CN108563468B (en) | Bluetooth sound box data processing method and device and Bluetooth sound box | |
| US20220309320A1 (en) | Almost-indirect convolution in quantized neural networks | |
| CN109598250B (en) | Feature extraction method, device, electronic equipment and computer readable medium | |
| CN116992946B (en) | Model compression method, device, storage medium and program product | |
| WO2024022060A1 (en) | Image registration method and apparatus, and storage medium | |
| US20210241105A1 (en) | Inference apparatus, inference method, and storage medium | |
| CN111582432B (en) | Network parameter processing method and device | |
| CN111027683A (en) | Data processing method, data processing device, storage medium and electronic equipment | |
| WO2023093260A1 (en) | Instruction processing apparatus and method, and computer device and storage medium | |
| CN119416823B (en) | Large model reasoning method, device, equipment and program product | |
| CN112416294B (en) | Processor, binary accumulation method thereof, and computer readable medium | |
| CN111556248B (en) | Shooting method, shooting device, storage medium and mobile terminal | |
| CN111382831B (en) | Method and device for accelerating forward reasoning of convolutional neural network model | |
| CN111324793B (en) | Method and device for controlling operations of storing data in a region of interest | |
| CN113128673B (en) | Data processing method, storage medium, neural network processor and electronic device | |
| CN107729145A (en) | Application program operation control method and device | |
| CN111833232A (en) | an image processing device | |
| CN112017100B (en) | Convolution operation method and related product | |
| CN113935279B (en) | Path-based optical simulation device activation method, device and terminal equipment | |
| KR102722476B1 (en) | Neural processing elements with increased precision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200417 |