WO2023103334A1 - Data processing method and apparatus of neural network simulator, and terminal - Google Patents

Abstract

The present application belongs to the technical field of data processing, and in particular relates to a data processing method and apparatus of a neural network simulator, and a terminal. The method comprises: acquiring instruction data; transporting data of a source end to a destination end in a transaction-level transportation manner and according to a first transportation parameter carried in a first transportation instruction; transporting, with cycle-level precision, data of the destination end to a cache according to a second transportation parameter carried in a second transportation instruction; and if the cache is not empty, executing a granularity operation on the basis of data in the cache, so as to obtain a cycle-level data processing result. Therefore, a cycle-level precise instruction operation and transaction-level fuzzy data transportation are realized, and an instruction of a neural network simulator can be calculated according to a cycle level, thus maintaining the consistency with hardware and precision, optimizing cycle-level dependency of data transportation, and reducing the complexity of the neural network simulator.

Description

Data processing method, device and terminal of a neural network simulator technical field

The present application belongs to the technical field of data processing, and in particular relates to a data processing method, device and terminal of a neural network simulator.

This application claims the priority of the Chinese patent application with the application number 202111494700.7 and the title of the invention "a data processing method, device and terminal for a neural network simulator" submitted to the China Patent Office on December 8, 2021, the entire content of which Incorporated in this application by reference.

Background technique

With the development of artificial intelligence big data technology, neural network simulators have shown significant advantages in processor microarchitecture design, TVM tool chain development and promotion, and RTL development verification.

However, with the large amount of data processed by the neural network processor, the multi-dimensional data, and the complex and diverse calculation methods, the current data processing methods of the neural network simulator can no longer meet the needs of use.

technical problem

The embodiment of the present application provides a data processing method, device, terminal and computer-readable storage medium of a neural network simulator, which can reduce the complexity of the neural network simulator, so that the neural network simulator can carry large data in the neural network processor , large computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification and other aspects play a significant role.

The first aspect of the embodiment of the present application provides a data processing method of a neural network simulator, including:

Acquiring instruction data; the instruction data includes a first transportation instruction carrying a first transportation parameter, a second transportation instruction carrying a second transportation parameter, and a granular operation instruction;

Transporting the data at the source end to the destination end in a transaction-level transport manner according to the first transport parameter carried by the first transport instruction;

Transporting the data at the destination to the cache with cycle-level precision according to the second transport parameter carried by the second transport instruction;

If the cache is not empty, a granular operation is performed based on the data in the cache to obtain a cycle-level data processing result.

According to some embodiments, based on the data processing method of a neural network simulator provided in the first aspect above, in the first possible implementation of the present application, the data at the source end is transferred to the destination end in a transaction-level transfer manner ,include:

Based on the communication handshake between the source end and the destination end, the data at the source end is transferred to the destination end in a transaction-level transfer manner.

According to some embodiments, based on the data processing method of a neural network simulator provided in the first aspect above, in the second possible implementation of the present application, the second handling parameters include the operation mode corresponding to the currently transported data;

The step of transferring the data of the destination to the cache according to the second transfer parameter carried by the second transfer instruction with cycle-level precision includes:

calculating the actual amount of data corresponding to the data at the destination end required for the granularity calculation according to the calculation mode corresponding to the currently transported data;

According to the real data volume and the data volume transferred from the source terminal to the destination terminal, it is judged whether the data volume of the destination terminal is greater than or equal to the real data volume required by the granular operation;

If the amount of data at the destination is greater than or equal to the actual amount of data required by the granularity calculation, the data at the destination is moved to the cache with cycle-level precision according to the second moving parameter.

According to some embodiments, based on the data processing method of a neural network simulator provided in the first aspect above, in the third possible implementation of the present application, the second handling parameters include cutting parameters and current handling data The corresponding operation mode;

The step of transferring the data of the destination to the cache according to the second transfer parameter carried by the second transfer instruction with cycle-level precision includes:

Cutting the data of the destination according to the cutting parameters in the second handling parameters according to cycle-level accuracy to obtain cutting data;

performing calculations on the cutting data according to the calculation mode corresponding to the currently transported data, to obtain the target data required for the granularity calculation;

Move the target data to the cache.

According to some embodiments, based on the data processing method of a neural network simulator provided in the first aspect above, and the first, second and third possible implementations above, in the fourth possible implementation of the present application In an implementation manner, when the target data required by the granular operation is part of the data in the matrix data of the destination end, the second transport parameter includes the first position coordinates corresponding to each data in the part of the data;

The transferring the data of the destination end to the cache according to the second transfer parameter with cycle-level accuracy includes:

The data corresponding to the first position coordinate in the matrix data of the destination end is transferred to the cache.

According to some embodiments, based on the data processing method of a neural network simulator provided in the first aspect above, and the first, second and third possible implementations above, in the fifth possible implementation of the present application In an implementation, when the target data required for the granularity calculation is the value transformed before winograd, the second transport parameter includes the second position coordinates of the data in the 4*4 data table required for the transformation before winograd ;

The transferring the data of the destination end to the cache according to the second transfer parameter with cycle-level accuracy includes:

Read the 4*4 data table stored at the destination according to the second position coordinates, and calculate the pre-winograd converted value based on the 4*4 data table, and convert the pre-winograd converted value The value is moved to the cache.

According to some embodiments, based on the data processing method of a neural network simulator provided in the first aspect above, and the first, second and third possible implementations above, in the sixth possible implementation of the present application In an implementation manner, before performing the granular operation based on the data in the cache to obtain cycle-level data processing results if the cache is not empty, the method includes:

Obtain the value of the winding flag bit of the cache, and the read address and write address of the cache;

Whether the cache is empty is judged according to whether the read address of the cache coincides with the write address and the value of the wrapping flag bit of the cache.

The second aspect of the embodiment of the present application provides a data processing device for a neural network simulator, including:

An acquisition unit, configured to acquire instruction data; the instruction data includes a first transportation instruction carrying a first transportation parameter, a second transportation instruction carrying a second transportation parameter, and a granular operation instruction;

The first transport unit is configured to transport the data at the source end to the destination end in a transaction-level transport manner according to the first transport parameters carried by the first transport instruction;

The second transfer unit is configured to transfer the data at the destination to the cache with cycle-level precision according to the second transfer parameters carried by the second transfer instruction;

The data processing unit is configured to, if the cache is not empty, perform granular operations based on the data in the cache to obtain cycle-level data processing results.

According to some embodiments, based on the data processing device of a neural network simulator provided in the second aspect above, in the first possible implementation manner of the present application, the first handling unit is also used for:

Based on the communication handshake between the source end and the destination end, the data at the source end is transferred to the destination end in a transaction-level transfer manner.

According to some embodiments, based on the data processing device of a neural network simulator provided in the second aspect above, in the second possible implementation of the present application, the second handling parameters include the operation mode corresponding to the currently transported data;

The second handling unit is also used for:

calculating the actual amount of data corresponding to the data at the destination end required for the granularity calculation according to the calculation mode corresponding to the currently transported data;

According to the real data volume and the data volume transferred from the source terminal to the destination terminal, it is judged whether the data volume of the destination terminal is greater than or equal to the real data volume required by the granular operation;

If the amount of data at the destination is greater than or equal to the actual amount of data required by the granularity calculation, the data at the destination is moved to the cache with cycle-level precision according to the second moving parameter.

According to some embodiments, based on the data processing device of a neural network simulator provided in the second aspect above, in the third possible implementation of the present application, the second handling parameters include cutting parameters and current handling data The corresponding operation mode;

The second handling unit is also used for:

Cutting the data of the destination according to the cutting parameters in the second handling parameters according to cycle-level accuracy to obtain cutting data;

performing calculations on the cutting data according to the calculation mode corresponding to the currently transported data, to obtain the target data required for the granularity calculation;

Move the target data to the cache.

According to some embodiments, based on the data processing device for a neural network simulator provided in the second aspect above, and the first, second and third possible implementations above, in the fourth possible implementation of the present application In an implementation manner, when the target data required by the granular operation is part of the data in the matrix data of the destination end, the second transport parameter includes the first position coordinates corresponding to each data in the part of the data;

The second handling unit is also used for:

The data corresponding to the first position coordinate in the matrix data of the destination end is transferred to the cache.

According to some embodiments, based on the data processing device for a neural network simulator provided in the first aspect above, and the first, second and third possible implementations above, in the fifth possible implementation of the present application In an implementation, when the target data required for the granularity calculation is the value transformed before winograd, the second transport parameter includes the second position coordinates of the data in the 4*4 data table required for the transformation before winograd ;

The second handling unit is also used for:

Read the 4*4 data table stored at the destination according to the second position coordinates, and calculate the pre-winograd converted value based on the 4*4 data table, and convert the pre-winograd converted value The value is moved to the cache.

According to some embodiments, based on the data processing method of a neural network simulator provided in the above-mentioned second aspect, and the above-mentioned first, second, and third possible implementations, in the sixth possible implementation of the present application In an implementation manner, the data processing unit is further configured to:

Before said if the cache is not empty, the granular operation is executed based on the data in the cache, and before the cycle-level data processing result is obtained, the value of the winding flag bit of the cache is obtained, and the The read address and write address of the cache;

Whether the cache is empty is judged according to whether the read address of the cache coincides with the write address and the value of the wrapping flag bit of the cache.

The third aspect of the embodiments of the present application provides a terminal, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the steps of the foregoing method are implemented.

A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the foregoing method are implemented.

In the embodiment of the present application, the data at the source end is transferred to the destination end in a transaction-level transfer manner, and the data at the destination end is transferred to the cache with cycle-level precision, and then executed based on the data in the cache and the granular operation instructions The granular operation makes the neural network simulator of the present application mix cycle-level and transaction-level simulator design methods, realize cycle-level accurate instruction operation and transaction-level fuzzy data transfer, and also make the neural network simulator Instructions can be calculated according to the cycle level, which maintains the consistency and accuracy with the hardware, and optimizes the cycle-level dependence of data handling, reduces the complexity of the neural network simulator, and enables the neural network simulator to be processed in the neural network It plays a significant role in the handling of large data of the device, large computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification, etc.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings that are required in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and thus It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

Fig. 1 is the realization flowchart of the data processing method of a kind of neural network simulator provided by the embodiment of the present application;

Fig. 2 is a schematic diagram of data cutting provided by the embodiment of the present application;

FIG. 3 is a schematic diagram of a first specific implementation flow chart of step 103 of a data processing method for a neural network simulator provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of a second specific implementation flow chart of step 103 of a data processing method for a neural network simulator provided in an embodiment of the present application;

Fig. 5 is a schematic diagram of matrix data handling provided by the embodiment of the present application;

FIG. 6 is a schematic diagram of judging whether the cache is empty provided by the embodiment of the present application;

7 is a schematic structural diagram of a data processing device of a neural network simulator provided in an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a terminal provided by an embodiment of the present application.

Embodiments of the present invention

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and/or components, but does not exclude one or more other features. , whole, step, operation, element, component and/or the presence or addition of a collection thereof.

It should also be understood that the terminology used in the specification of the present application is for the purpose of describing specific embodiments only and is not intended to limit the present application.

A neural network simulator is a technical tool that provides modeling or some sort of research prototype for an artificial neural network. In general, neural network simulators are a resource for researchers to study how neural networks work. Different kinds of data collection help the simulator evaluate what's going on inside the artificial neural network. To effectively show researchers how neural networks work, neural network simulators often include versatile visual interfaces that display data graphically. Many of these have multiple windows that can be labeled as data modules for easy identification.

Traditional neural network simulators use cycle-level (cycle-level) modeling or transaction-level modeling. However, pure cycle-level modeling has a significant impact on the modeling complexity, running speed, modeling cycle, and tool chain usage of neural network processors; while pure transaction-level modeling can only be used for approximate simulation and pre-evaluation, Can't really do the same with actual hardware. Both of these two neural network simulators cannot meet the needs of neural network processors that are gradually changing in the direction of processing large amounts of data, multiple data dimensions, and complex and diverse calculation methods.

Based on this, the embodiment of the present application provides a data processing method, device, and terminal for a neural network simulator, which mix cycle-level and transaction-level simulator design methods, and can realize cycle-level accurate instruction operations and transaction-level fuzzy Data handling maintains the consistency and accuracy of the hardware, and can play a significant role in the neural network processor's big data handling, large computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification, etc.

In order to illustrate the technical solution of the present application, specific examples are used below to illustrate.

FIG. 1 shows a schematic flowchart of a data processing method for a neural network simulator provided by an embodiment of the present application. The method is applied to a terminal and can be executed by a data processing device of a neural network simulator configured on the terminal. Wherein, the above-mentioned terminal may be an intelligent terminal such as a computer or a server. The data processing method of above-mentioned neural network simulator can comprise step 101 to step 104, detailed description is as follows:

Step 101, acquire instruction data.

In the embodiment of the present application, the above instruction data may include a first transportation instruction carrying a first transportation parameter, a second transportation instruction carrying a second transportation parameter, and a granular operation instruction. Moreover, the above instruction data may be instruction data acquired by different modules in the neural network simulator from the instruction control flow.

For example, the above-mentioned first handling instruction is the instruction data obtained by the exdma module in the neural network simulator from the instruction control flow, and the above-mentioned second handling instruction is the instruction data obtained by the xdma module in the neural network simulator from the instruction control flow, The above granular operation instruction is the instruction data obtained by the granular operation module cube in the neural network simulator from the instruction control flow.

The first transfer instruction is used to transfer the data at the source end to the destination end in a transaction-level transfer manner, and the second transfer instruction is used to transfer the data at the destination end to the cache with cycle-level precision.

The above-mentioned source and destination can be different modules in the neural network simulator, for example, the above-mentioned source and destination can be the exdma module in the neural network simulator, such as eidma module and eodma, and the above-mentioned destination can be a neural network simulation xdma modules in the device, such as idma modules and odma.

In some embodiments of the present application, according to some embodiments, the above-mentioned first transport parameter may include the data amount of data transported from the source end to the destination end each time, that is, the handshake granularity, for example, 1Kb or 5Kb, and the current The total amount of data that needs to be moved.

The above-mentioned second transport parameters may include cutting parameters and an operation mode corresponding to the current transport data.

Wherein, the above operation modes may include a winograd operation mode, a matrix operation mode, a padding mode padding, a 0 insertion mode deconvlution, and an address jump mode dilated.

The above cutting parameters can include cutting parameters H, W, D for cutting in three directions of data height (zeta), width (epsilon=ci) and depth (dense), as well as dense times, zeta times, epsilon times, kernel dense dimensions The number of sliding windows, winograd_loop16, weight parameter multiplexing times and other data. Among them, when the operation mode corresponding to the currently transported data is the winograd operation mode, winograd_loop16=16 means 16 piexl cycles, and when the operation mode corresponding to the currently transported data is the non-winograd operation mode, winograd_loop16=1.

For example, as shown in Figure 2, when H=16, W=32, and D=8, data ci0 can be obtained by cutting the data at the destination, and data ci0 to data ci7 can be obtained sequentially by cutting along the direction of ci.

In addition, the data at the source end may be image data or parameter data, which is not limited in this application.

Step 102: Transport the data at the source end to the destination end in a transaction-level transport manner according to the first transport parameter carried in the first transport instruction.

In the embodiment of this application, moving the data from the source end to the destination end belongs to the data preparation process, which uses transaction-level handling, and belongs to loosely coupled handling, which can be independent of the period-level handling process, that is, It is independent from the cycle-level transfer process of transferring the data of the destination to the cache, so that the data of the destination can be prepared in advance, reducing the waiting time when transferring the data of the destination to the cache.

According to some embodiments, in some implementations of the present application, in the above step 102, moving the data at the source end to the destination end in a transaction-level transfer manner may include: based on the communication handshake between the source end and the destination end, transferring the Data is moved to the destination using transaction-level handling.

In this embodiment of the application, the communication handshake between the source and the destination means that the destination needs to wait for the source to write the data into The destination, and after the source writes the data to the destination, it needs to notify the destination that the data is ready.

Specifically, before the destination end reads data from its own storage space dm, it sends a waiting signal dest_wo_src to the source end, waiting for the source end to write data into dm of the destination end. After the source end writes the data to the destination end, it sends the enable signal src_ub_dest to the destination end, and the destination end reads the data from dm, and after completing the data reading, sends the enable signal dest_ub_src to the source end to notify the source end Data space has been freed.

It should be noted that, in some embodiments of the present application, before the destination end reads data from dm, it can continue to send the waiting signal dest_wo_src to the source end, and the source end performs cumulative counting, and stores the data in dm in advance, while There is no need to wait for the destination end to read the data in dm before continuing to send the enable signal dest_wo_src to the source end, which reduces the waiting time for data reading and improves the efficiency of data storage and reading.

Step 103, according to the second transfer parameter carried by the second transfer instruction, the data at the destination is transferred to the cache with cycle-level precision.

In the embodiment of the present application, the cache refers to the storage space corresponding to the granular operation. The above-mentioned transfer of destination data to the cache belongs to cycle-level transfer and calculation.

Specifically, in some implementations of the present application, as shown in FIG. 3 , in the above step 103, the data at the destination is transferred to the cache with cycle-level precision according to the second transfer parameter carried by the second transfer instruction, It may include steps 301 to 303 as follows.

Step 301 , according to the cutting parameters in the second transport parameters, cutting the data of the destination end with cycle-level accuracy to obtain cutting data.

As shown in the above-mentioned step 101, the above-mentioned cutting parameters may include cutting parameters H, W, D for cutting data from three directions of height (zeta), width (epsilon=ci) and depth (dense), as well as dense times, zeta Times, epsilon times, sliding window times in the kernel dense dimension, winograd_loop16, weight parameter reuse times and other data.

Step 302: Perform calculations on the cutting data according to the calculation mode corresponding to the currently transported data to obtain the target data required for the granularity calculation.

In the embodiment of the present application, the above operation modes may include padding in padding mode, deconvlution in 0 insertion mode, and dilated in jump address mode.

In the above step 302, the cutting data is calculated according to the calculation mode corresponding to the currently transported data, and the target data required for the granularity calculation is obtained may include the following situations: if the calculation mode corresponding to the currently transported data is the deconvlution calculation mode , the cutting data is calculated by inserting 0 to obtain the target data required for the granularity calculation; if the operation mode corresponding to the currently transported data is dilated, the data after the jump address is calculated for the cutting data to obtain The target data required by the granular operation.

Step 303, moving the target data to the cache.

In the embodiment of the present application, after calculating and obtaining the target data required by the granularity calculation in the process of transporting data, the target data is directly transported to the cache, without intermediate cache.

Step 104, if the cache is not empty, perform granular operations based on the data in the cache to obtain cycle-level data processing results.

In the embodiment of the present application, when the cache is not empty, it means that the data used for the granular operation has been prepared, therefore, the granular operation can be directly performed based on the data in the cache, that is, the cube operation, the The calculation is accurate calculation at the cycle level, therefore, the output of cycle-level accurate results is realized, which can be used for RTL data calculation comparison.

In the embodiment of the present application, the data at the source end is transferred to the destination end in a transaction-level transfer manner, and the data at the destination end is transferred to the cache with cycle-level precision, and then executed based on the data in the cache and the granular operation instructions The granular operation makes the neural network simulator of this application mix cycle-level and transaction-level simulator design methods, realize cycle-level accurate instruction operation and transaction-level fuzzy data transfer, and maintain consistency with hardware And accuracy, it can play a great role in neural network processor big data handling, large computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification, etc.

Exemplarily, when the above-mentioned neural network simulator is a neural network simulator for simulating an artificial neural network model for face recognition, the data at the source end may be face image data, and the first transfer instruction carried by the above-mentioned A transfer parameter may include the data volume of the face image data, the first transfer instruction is used to transfer the face image data at the source end to the destination end in a transaction-level transfer manner, and the second transfer parameter carried by the second transfer instruction may be Including the cutting parameters H0, W0, and D0 for cutting the data height (zeta), width (epsilon=ci) and depth (dense) of the face image data, which are used to cut the face image data of the destination end by cycle Cutting with high-level precision to obtain cutting data, so as to transfer the face image data of the destination to the cache with cycle-level precision, and when the cache is not empty, perform granular operations based on the data in the cache, that is, Cube operation, and finally get cycle-level face recognition results.

Wherein, the above-mentioned cube operation refers to using a neural network algorithm to perform operations on the face image data in the cache to obtain a face classification result. For example, the face recognition result corresponding to the above-mentioned face image data is Zhang San's face, or Li Four faces.

The above neural network algorithm can include Layer Cubing algorithm (layer-by-layer algorithm), By-layer Spark Cubing algorithm, Fast(in-mem) Cubing algorithm, that is, "by segment" (By Segment) or "by block" (By Split) algorithm and other neural network algorithms, this application does not limit it.

This application transfers the face image data at the source end to the destination end in a transaction-level transfer manner, and transfers the face image data at the destination end to the cache with cycle-level accuracy, and then based on the face image data in the cache and the particles High-precision operation instructions execute granular operations, so that the neural network simulator of this application mixes cycle-level and transaction-level simulator design methods, and realizes cycle-level accurate instruction operations and transaction-level fuzzy data transfer. In the process of realizing face image recognition, the model maintains the consistency and accuracy with the hardware, and can play a role in neural network processor big data handling, large computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification, etc. great effect.

It should also be noted that the above-mentioned neural network simulator can also be a neural network simulator for simulating the working process of an artificial neural network model in other application scenarios, for example, the above-mentioned neural network simulator can also be a neural network simulator for simulating the A simulator of the working process of the artificial neural network model for recognition, obstacle recognition or animal classification, the present application does not limit this.

It should also be noted that, besides image data, the above-mentioned data at the source end may also be different types of data such as voice data, and this application does not limit the data type of the data at the source end.

According to some embodiments, when the data at the source end is speech data, correspondingly, the above-mentioned neural network simulator may be a neural network simulator for analyzing and processing speech data, for example, the above-mentioned neural network simulator may be a A neural network simulator for processing speech data such as classification, denoising, etc.

According to some embodiments, in some implementations of the present application, in the above step 103, in the process of transferring the data of the destination end to the cache with cycle-level precision, the amount of data transferred from the source end to the destination end and the granularity calculation can be performed first. The required real data volume is synchronized to determine whether there is data that can be transported at the destination, and then the data is transported. Specifically, as shown in FIG. accomplish.

Step 401: Calculate the real data amount corresponding to the data at the destination end required by the granularity calculation according to the calculation mode corresponding to the currently transported data.

In the embodiment of the present application, the operation mode corresponding to the currently transferred data may include winograd operation mode, matrix operation mode, padding mode padding, 0 insertion mode deconvlution, and jump address mode dilated.

Since the target data required for granular operations may be obtained according to the operation mode corresponding to the currently transferred data, rather than directly the data in the destination end, the data corresponding to the target data required for granular operations The actual amount of data corresponding to the data at the destination end required for granular computing may be inconsistent. For example, the amount of data corresponding to the target data required by the granular operation is greater than the actual data amount corresponding to the data of the destination end required by the granular operation. Moreover, since the data volume of the target data is known and is a fixed data volume, the actual data corresponding to the data at the destination end required by the granularity calculation can be inversely deduced according to the calculation mode corresponding to the currently transported data. The amount of data.

For example, when the operation mode corresponding to the currently transported data is the zero-insertion mode, the actual data volume corresponding to the destination data required for the granular operation can be calculated according to the operation mode and the data volume of the target data.

Step 402: According to the real data volume and the data volume transported from the source to the destination, determine whether the data volume at the destination is greater than or equal to the real data volume required by the granular operation.

In the embodiment of the present application, by handshaking the calculated real data volume with the data volume transported by the source terminal, it is determined whether the data volume transported by the source terminal reaches the real data volume required by the granularity calculation, and if not , waiting for the source to move the data to the destination.

Step 403, if the amount of data at the destination is greater than or equal to the actual amount of data required by the granularity calculation, then move the data at the destination to the cache with cycle-level precision according to the second moving parameter.

When the amount of data at the destination meets the actual data amount required by the granular operation, it means that the data amount at the destination has reached the actual data amount required by the granular operation, and thus data transfer can be performed.

In the embodiment of the present application, since it is only necessary to transfer the data of the destination end to the cache with cycle-level precision, the amount of data transferred from the source end to the destination end and the actual data amount required for granular calculations need to be synchronized once That is, there is no need for cycle-level synchronization of intermediate processes, so that the transaction-level process of moving the data from the source to the destination can be independent of the cycle-level process of moving the data from the destination to the cache, and the data at the destination can be Prepare in advance to reduce the waiting time when moving the destination data to the cache. In addition, in the process of data transfer in the embodiment of the present application, while calculating padding, deconvlution, dilated and other modes that need to jump addresses or fill data, while transferring dm real and valid data, avoiding access to ddr cache and bandwidth in the middle of the data, and improving data The efficiency of handling.

According to some embodiments, in some implementations of the present application, the data at the source end can also be moved to the destination end by using a transaction-level transfer method, and when the data at the destination end is transferred to the cache with cycle-level precision, through Abstract computing-dependent datasets in different modes, such as abstract transpose computing mode and winograd computing-dependent datasets, to reduce system complexity Operation-level iteration of complex signals in RTL cycle-level scenarios, reducing cycle-level modeling complexity At the same time, the performance of the neural network simulator is greatly improved.

Specifically, in some implementations of the present application, when the target data required for the above-mentioned granularity operation is part of the data in the matrix data of the destination end, the above-mentioned second transport parameter may include the corresponding first position coordinates; in the above step 103, transporting the data of the destination end to the cache according to the cycle-level precision according to the second transfer parameters may include: transferring the data corresponding to the first position coordinates in the matrix data of the destination end Moved to the cache.

For example, if the target data required for the granular operation is the first row of data after the transposition of the matrix data at the destination, the above-mentioned second handling parameters may include the corresponding first row of each data in this row of data in the matrix data of the destination. Position coordinates, and then select this row of data from the matrix data at the destination end according to the first position coordinates, and move it to the cache.

Specifically, as shown in Figure 5, when the matrix data at the destination is 16*16 matrix data, if the target data required for the granularity operation is the first column data of the matrix data, the first position coordinates can be used to select This column contains 16 data and moves it to the cache.

It should be noted that, because the traditional neural network simulator, its data transfer method needs to be synchronized with the hardware, therefore, it is impossible to abstract the matrix data containing multiple rows and columns according to the operation mode corresponding to the currently transferred data to obtain the The data that the operation mode depends on, or the data needs to be obtained through complex operation iterations. For example, for the above-mentioned 16*16 matrix data, it is impossible to select the 16 data contained in this column through the first position coordinates. Therefore, There are problems of high complexity of data operation and high system complexity.

This application circulates the parameter matrix by inputting data instructions, and transposes special rows in the matrix, for example, by transposing the coordinates (x, y) of the data in the matrix, making x=y, y=x, to obtain the required granularity operation data, without data synchronization with hardware, decoupling data handling, effectively reducing system complexity for RTL Operation-level iteration of complex signals in cycle-level scenarios reduces the complexity of cycle-level modeling and greatly improves the performance of neural network simulators.

According to some embodiments, in some implementations of the present application, when the target data required for the above-mentioned granularity calculation is the value transformed before winograd, the second handling parameter may include the 4*4 data table required for the transformation before winograd The second location coordinates of the data in .

In the above step 103, transferring the data of the destination end to the cache according to the second transfer parameters with cycle-level accuracy may include: reading the 4*4 data table stored at the destination end according to the second position coordinates , and calculate the pre-winograd transformed value based on the 4*4 data table, and transfer the pre-winograd transformed value to the cache.

For example, as shown in Table 1 below, according to the coordinates of the first data d0-d2-d8+d10 in the 4*4 data table required for winograd transformation, translate 3 coordinates down and to the right to get 4*4 The data table, and based on the 4*4 data table, calculate the converted value before winograd.

Table I:

It should be noted that due to the traditional neural network simulator, its data handling method needs to be synchronized with the hardware, and only one piece of data in the 4*4 data table can be read at a time, and the destination end cannot be read according to the second position coordinates. The entire 4*4 data table is stored, or the data needs to be obtained through complex calculation iterations. Therefore, there are problems of high complexity of data calculation and high system complexity.

In this application, according to the coordinates of the first data d0-d2-d8+d10 in the 4*4 data table required for winograd transformation, the 4*4 data table is obtained by shifting 3 coordinates downward and to the right, without Handshaking with the hardware cache can decouple data handling and effectively reduce system complexity for RTL Operation-level iteration of complex signals in cycle-level scenarios reduces the complexity of cycle-level modeling and greatly improves the performance of neural network simulators.

The neural network simulator instruction set of this application is calculated according to the cycle level, which maintains the consistency and accuracy with the hardware. The data transfer adopts the transaction-level transfer, and through the basic handshake granularity, abstracts the acceleration of different modes such as transposition and winograd algorithm , padding, deconvlution, dilated and other scene calculations rely on data sets to reduce system complexity and iterate on scratchpad transfer levels for cycle-level scenarios with complex signal operations, reducing the complexity of cycle-level modeling and greatly improving system performance.

According to some embodiments, in some implementations of the present application, before the above step 104, it may be checked whether the cache is empty.

Specifically, it is possible to determine whether the cache is empty by obtaining the value of the winding flag bit of the cache and whether the read address and the write address coincide. If it is empty, wait for data to be written. If not, then based on the cache Execute the granularity operation on the data in to obtain cycle-level data processing results. Among them, the cache adopts the first-in-first-out data reading and writing method, and the value of the winding flag can be obtained through the reading and writing interaction between the cube computing module and the destination.

For example, when the destination end writes data to the cache and the cube operation reads data from the cache, as shown in (a) in Figure 6, when the winding flag bit ring_flag = 0, it means that the cache reads and writes In the case of non-winding, if the read and write addresses are the same, it means that the cache is empty. As shown in (b) of FIG. 6, when the winding flag bit ring_flag=1, it indicates that the reading and writing of the cache has generated winding. In this case, the read and write addresses are the same, indicating that the cache is full.

In the embodiment of the present application, the value of the above winding flag and the read address and write address of the cache are used to determine whether the cache is empty, which simplifies the handshake process of the cycle-level cube operation, improves the operation efficiency, and reduces the neural network simulator. The complexity makes the cycle-level cube operation and hardware settings independent of each other.

According to some embodiments, in some implementations of the present application, the granular operation is performed based on the data in the cache, and after obtaining the cycle-level data processing result, the result can also be stored in the destination end and provided to other Granular operations.

In the embodiment of the present application, the data at the source end is transferred to the destination end in a transaction-level transfer manner, and the data at the destination end is transferred to the cache with cycle-level precision, and then executed based on the data in the cache and the granular operation instructions The granular operation makes the neural network simulator of the present application mix cycle-level and transaction-level simulator design methods, realize cycle-level accurate instruction operation and transaction-level fuzzy data transfer, and make the neural network simulator instructions It can be calculated according to the cycle level, which maintains the consistency and accuracy with the hardware, and also optimizes the cycle-level dependence of data transfer, reduces the complexity of the neural network simulator, and can handle large data and large calculations in the neural network processor. It plays a significant role in power characteristics, architecture evaluation, instruction set tool chain development, RTL verification, etc.

It should be noted that for the foregoing method embodiments, for the sake of simple description, they are expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence. Because according to the present invention, certain steps can be carried out in other orders.

FIG. 7 shows a schematic structural diagram of a data processing device 700 for a neural network simulator provided by an embodiment of the present application, including an acquisition unit 701 , a first handling unit 702 , a second handling unit 703 and a data processing unit 704 .

An acquisition unit 701, configured to acquire instruction data; the instruction data includes a first transportation instruction carrying a first transportation parameter, a second transportation instruction carrying a second transportation parameter, and a granular operation instruction;

The first transfer unit 702 is configured to transfer the data at the source end to the destination end in a transaction-level transfer manner according to the first transfer parameters carried in the first transfer instruction;

The second transfer unit 703 is configured to transfer the data at the destination to the cache with cycle-level precision according to the second transfer parameters carried by the second transfer instruction;

The data processing unit 704 is configured to execute the granular operation based on the data in the cache if the cache is not empty, and obtain a cycle-level data processing result.

In some embodiments of the present application, the above-mentioned first handling unit 702 is also used for:

Based on the communication handshake between the source end and the destination end, the data at the source end is transferred to the destination end in a transaction-level transfer manner.

In some implementations of the present application, the second transport parameters include the calculation mode corresponding to the currently transported data; the above-mentioned second transport unit 703 is also used to:

calculating the actual amount of data corresponding to the data at the destination end required for the granularity calculation according to the calculation mode corresponding to the currently transported data;

According to the real data volume and the data volume transferred from the source terminal to the destination terminal, it is judged whether the data volume of the destination terminal is greater than or equal to the real data volume required by the granular operation;

If the amount of data at the destination is greater than or equal to the actual amount of data required by the granularity calculation, the data at the destination is moved to the cache with cycle-level precision according to the second moving parameter.

In some embodiments of the present application, the second transport parameters include cutting parameters and calculation modes corresponding to the current transport data;

The above-mentioned second handling unit 703 is also used for:

Cutting the data of the destination according to the cutting parameters in the second handling parameters according to cycle-level accuracy to obtain cutting data;

performing calculations on the cutting data according to the calculation mode corresponding to the currently transported data, to obtain the target data required for the granularity calculation;

Move the target data to the cache.

In some embodiments of the present application, when the target data required by the granularity operation is part of the data in the matrix data of the destination end, the second transport parameter includes the first data corresponding to each data in the part of the data. 1. Position coordinates; the above-mentioned second handling unit 703 is also used for:

The data corresponding to the first position coordinate in the matrix data of the destination end is transferred to the cache.

In some embodiments of the present application, when the target data required by the granularity operation is the value transformed before winograd, the second handling parameter includes the data in the 4*4 data table required by the transformation before winograd The second position coordinates; the above-mentioned second handling unit 703 is also used for:

Read the 4*4 data table stored at the destination according to the second position coordinates, and calculate the pre-winograd converted value based on the 4*4 data table, and convert the pre-winograd converted value The value is moved to the cache.

In some embodiments of the present application, the above-mentioned data processing unit is also used for:

Before said if the cache is not empty, the granular operation is executed based on the data in the cache, and before the cycle-level data processing result is obtained, the value of the winding flag bit of the cache is acquired, and the The read address and write address of the cache;

Whether the cache is empty is judged according to whether the read address of the cache coincides with the write address and the value of the wrapping flag bit of the cache.

It should be noted that, for the convenience and brevity of the description, the specific working process of the data processing device 700 of the above-described neural network simulator can refer to the corresponding process of the method described in the above-mentioned Figures 1 to 6, and will not be repeated here. .

As shown in FIG. 8 , the present application provides a terminal for realizing the data processing method of the above-mentioned neural network simulator. The terminal 8 may include: a processor 80, a memory 81, and stored in the memory 81 and can be used in the A computer program 82 running on the processor 80, such as a memory allocation program. When the processor 80 executes the computer program 82, it realizes the steps in the above-mentioned embodiments of the data processing method of each neural network simulator, such as steps 101 to 104 shown in FIG. 1 . Alternatively, when the processor 80 executes the computer program 82, it realizes the functions of the modules/units in the above device embodiments, such as the functions of the units 701 to 704 shown in FIG. 7 .

The computer program can be divided into one or more modules/units, and the one or more modules/units are stored in the memory 81 and executed by the processor 80 to complete the present application. The one or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal. For example, the computer program can be divided into an acquisition unit, a first handling unit, a second handling unit and a data processing unit, and the specific functions of each unit are as follows:

An acquisition unit, configured to acquire instruction data; the instruction data includes a first transportation instruction carrying a first transportation parameter, a second transportation instruction carrying a second transportation parameter, and a granular operation instruction;

The first transport unit is configured to transport the data at the source end to the destination end in a transaction-level transport manner according to the first transport parameters carried by the first transport instruction;

The second transfer unit is configured to transfer the data at the destination to the cache with cycle-level precision according to the second transfer parameters carried by the second transfer instruction;

The data processing unit is configured to execute the granular operation based on the data in the cache if the cache is not empty, to obtain a cycle-level data processing result.

The terminal may be a computing device such as a computer or a server. The terminal may include, but not limited to, a processor 80 and a memory 81 . Those skilled in the art can understand that FIG. 8 is only an example of a terminal, and does not constitute a limitation on the terminal. It may include more or less components than those shown in the figure, or combine certain components, or different components, such as the Terminals may also include input and output devices, network access devices, buses, and so on.

The so-called processor 80 may be a central processing unit (Central Processing Unit, CPU), can also be other general-purpose processors, digital memory allocators (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

The storage 81 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. The memory 81 may also be an external storage device of the terminal, such as a plug-in hard disk equipped on the terminal, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc. Further, the memory 81 may also include both an internal storage unit of the terminal and an external storage device. The memory 81 is used to store the computer program and other programs and data required by the terminal. The memory 81 can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used for illustration. In practical applications, the above-mentioned functions can be assigned to different functional units, Completion of modules means that the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit, and the above-mentioned integrated units may adopt hardware It can also be implemented in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working processes of the units and modules in the above system, reference may be made to the corresponding processes in the aforementioned method embodiments, and details will not be repeated here.

In the above-mentioned embodiments, the descriptions of each embodiment have their own emphases, and for parts that are not detailed or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

In the embodiments provided in this application, it should be understood that the disclosed device/terminal and method may be implemented in other ways. For example, the device/terminal embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or Components may be combined or integrated into another system, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated module/unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments in the present application can also be completed by instructing related hardware through computer programs. The computer programs can be stored in a computer-readable storage medium, and the computer When the program is executed by the processor, the steps in the above-mentioned various method embodiments can be realized. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (Read-Only Memory, ROM) , Random Access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, computer-readable media Excludes electrical carrier signals and telecommunication signals.

The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still implement the foregoing embodiments Modifications to the technical solutions described in the examples, or equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the application, and should be included in the Within the protection scope of this application.

Claims (10)

A data processing method for a neural network simulator, comprising: Acquiring instruction data; the instruction data includes a first transportation instruction carrying a first transportation parameter, a second transportation instruction carrying a second transportation parameter, and a granular operation instruction; Transporting the data at the source end to the destination end in a transaction-level transport manner according to the first transport parameter carried by the first transport instruction; Transporting the data at the destination to the cache with cycle-level precision according to the second transport parameter carried by the second transport instruction; If the cache is not empty, a granular operation is performed based on the data in the cache to obtain a cycle-level data processing result. The data processing method of the neural network simulator according to claim 1, wherein said transferring the data at the source end to the destination end in a transaction-level transfer mode includes: Based on the communication handshake between the source end and the destination end, the data at the source end is transferred to the destination end in a transaction-level transfer manner. The data processing method of neural network simulator as claimed in claim 1, is characterized in that, described second handling parameter comprises the computing mode corresponding to the data of current handling; The step of transferring the data of the destination to the cache according to the second transfer parameter carried by the second transfer instruction with cycle-level precision includes: calculating the actual amount of data corresponding to the data at the destination end required for the data processing according to the operation mode corresponding to the currently transported data; According to the real data volume and the data volume transferred from the source terminal to the destination terminal, it is judged whether the data volume of the destination terminal is greater than or equal to the real data volume required by the granular operation; If the amount of data at the destination is greater than or equal to the actual amount of data required by the granularity calculation, the data at the destination is moved to the cache with cycle-level precision according to the second moving parameter. The data processing method of the neural network simulator according to claim 1, wherein the second handling parameters include cutting parameters and the corresponding operation mode of the current handling data; The step of transferring the data of the destination to the cache according to the second transfer parameter carried by the second transfer instruction with cycle-level precision includes: Cutting the data of the destination according to the cutting parameters in the second handling parameters according to cycle-level accuracy to obtain cutting data; performing calculations on the cutting data according to the calculation mode corresponding to the currently transported data, to obtain the target data required for the granularity calculation; Move the target data to the cache. The data processing method of the neural network simulator according to any one of claims 1-4, wherein when the target data required for the granularity operation is part of the data in the matrix data of the destination, the second The second handling parameter includes the first position coordinates corresponding to each data in the partial data; The transferring the data of the destination end to the cache according to the second transfer parameter with cycle-level accuracy includes: The data corresponding to the first position coordinate in the matrix data of the destination end is transferred to the cache. The data processing method of a neural network simulator according to any one of claims 1-4, wherein when the required target data for the granularity calculation is the value transformed before winograd, the second transport parameter Including the second position coordinates of the data in the 4*4 data table required for winograd transformation; The transferring the data of the destination end to the cache according to the second transfer parameter with cycle-level accuracy includes: Read the 4*4 data table stored at the destination according to the second position coordinates, and calculate the pre-winograd converted value based on the 4*4 data table, and convert the pre-winograd converted value The value is moved to the cache. The data processing method of the neural network simulator according to any one of claims 1-4, wherein, if the cache is not empty, the granularity calculation is performed based on the data in the cache to obtain Before cycle-level data processing results, including: Obtain the value of the winding flag bit of the cache, and the read address and write address of the cache; Whether the cache is empty is judged according to whether the read address of the cache coincides with the write address and the value of the wrapping flag bit of the cache. A data processing device for a neural network simulator, characterized in that it comprises: An acquisition unit, configured to acquire instruction data; the instruction data includes a first transportation instruction carrying a first transportation parameter, a second transportation instruction carrying a second transportation parameter, and a granular operation instruction; The first transport unit is configured to transport the data at the source end to the destination end in a transaction-level transport manner according to the first transport parameters carried by the first transport instruction; The second transfer unit is configured to transfer the data at the destination to the cache with cycle-level precision according to the second transfer parameters carried by the second transfer instruction; The data processing unit is configured to, if the cache is not empty, perform granular operations based on the data in the cache to obtain cycle-level data processing results. A terminal, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, characterized in that, when the processor executes the computer program, the computer program according to claims 1 to 7 is implemented. The steps of any one of the described methods. A computer-readable storage medium storing a computer program, wherein the computer program implements the steps of the method according to any one of claims 1 to 7 when the computer program is executed by a processor.