CN114327630A - High-performance operator generation method suitable for Huaji Shengteng chip - Google Patents

Abstract

The present invention discloses a high performance operator generation method suitable for Huazhi rising chip. Wherein, the method comprises the following steps: generating a plurality of candidate operation functions in a target development mode, wherein the target development mode is a tensor iterator kernel development mode determined by a tensor acceleration engine operator development framework based on the rising artificial intelligence processor; selecting a target operation function to be used from a plurality of candidate operation functions; and executing the target operation by using the target operation function and the target operation data to obtain a target operation result. The invention solves the technical problem of low development efficiency of high-performance operators in the related technology.

Description

A high-performance operator generation method suitable for Huawei Ascend chips

technical field

The present application relates to the field of artificial intelligence, and in particular, to a high-performance operator generation method suitable for Huawei Ascend chips.

Background technique

High-performance operators are computational functions involved in deep learning models. Common operators include convolution, matrix multiplication, and Rectified Linear Unit (ReLU). As a basic part of the artificial intelligence (Artificial Intelligence, AI) computing framework, high-performance operators call down the central processing unit (Central Processing Unit, CPU), graphics processing unit (Graphics Processing Unit, GPU), neural network processor (Neural- AI chips such as network Processing Unit (NPU) provide operation interfaces for various computing frameworks. High-performance operators are an important foundation for fully exploiting the computing potential of chips and improving the efficiency of training and inference.

Among related technologies, the Ascend technology stack provides a Tensor Boost Engine (TBE) operator development framework, and developers can choose to use a Domain-Specific Language (DSL) or a tensor iterator kernel (Tensor Iterator Kernel, TIK) development method for operator development. Among them, DSL has poor flexibility and performance, but high development efficiency; TIK has high flexibility and performance, but low development efficiency and large development workload. Developers are required to be familiar with the underlying hardware architecture and manually plan operator scheduling. When using the existing technology for operator development, it is difficult to obtain good operator performance and development efficiency at the same time.

For the above problems, no effective solution has been proposed yet.

SUMMARY OF THE INVENTION

The embodiment of the present application provides a high-performance operator generation method suitable for Huawei Ascend chips, so as to at least solve the technical problem of low development efficiency of high-performance operators in the related art.

According to one of the embodiments of the present application, a high-performance operator generation method suitable for Huawei Ascend chips is provided, including: generating a plurality of candidate operation functions in a target development mode, wherein the target development mode is based on the Ascend The tensor iterator kernel development method determined by the tensor acceleration engine operator development framework of the artificial intelligence processor; select the target operation function to be used from multiple candidate operation functions; use the target operation function and the target operation data to perform the target operation, and obtain Target operation result.

Optionally, selecting the target operation function from the multiple candidate operation functions includes: selecting the data handling function from the multiple candidate operation functions.

Optionally, using the target operation function and the target operation data to perform the target operation, and obtaining the target operation result includes: obtaining the target operation data, wherein the target operation data includes: a first source operand, a first destination operand and a data length; using The data moving function, the first source operand, the first destination operand and the data length perform a data moving operation to obtain a data moving result.

Optionally, using the data handling function, the first source operand, the first destination operand, and the data length to perform the data handling operation, and obtaining the data handling result includes: using the data length to perform block processing on the first source operand, and obtaining a score. Block result; when it is determined that there is no tail block based on the block result, the first source operand is transferred to the first destination operand through the data transfer function and the block result, and the data transfer result is obtained; when it is determined based on the block result that there is a tail block When block, determine the target transfer mode according to the storage location of the first source operand and the first destination operand, and transfer the first source operand to the first destination operand through the data transfer function and the target transfer mode, and obtain the data transfer result .

Optionally, selecting the target operation function from the multiple candidate operation functions includes: selecting the precision vector calculation function from the multiple candidate operation functions.

Optionally, using the target operation function and the target operation data to perform the target operation, and obtaining the target operation result includes: obtaining the target operation data, wherein the target operation data includes: the second source operand, the second destination operand and the first instruction name. ; Use the precision vector calculation function, the second source operand, the second destination operand and the first instruction name to perform the precision vector calculation operation to obtain the precision vector calculation result.

Optionally, using the precision vector calculation function, the second source operand, the second destination operand and the first instruction name to perform the precision vector calculation operation, and obtaining the precision vector calculation result includes: when the second source operand is not located in the global memory. , determine the number of precision vector calculations, use the precision vector calculation function and the first instruction name to perform the precision vector calculation operation on the second source operand to obtain the precision vector calculation result, and transfer the precision vector calculation result to the second destination operand; when When the second source operand is located in the global memory, perform multi-core optimization processing on the second source operand to obtain the optimization processing result, determine the number of precision vector calculations, and use the precision vector calculation function and the first instruction name to perform precision vector calculation on the optimization processing result. Operate to obtain the precision vector calculation result, and transfer the precision vector calculation result to the second destination operand.

Optionally, selecting the target operation function from the multiple candidate operation functions includes: selecting a reduction vector calculation function from the multiple candidate operation functions.

Optionally, using the target operation function and the target operation data to perform the target operation, and obtaining the target operation result includes: obtaining the target operation data, wherein the target operation data includes: a third source operand, a third destination operand and a first data amount. ; Use the reduction vector calculation function, the third source operand, the third destination operand and the first data amount to perform multiple rounds of iterative processing to obtain the reduction vector calculation result.

Optionally, selecting the target operation function from the multiple candidate operation functions includes: selecting a data filling function from the multiple candidate operation functions.

Optionally, using the target operation function and the target operation data to perform the target operation, and obtaining the target operation result includes: obtaining the target operation data, wherein the target operation data includes: the fourth source operand, the fourth destination operand and the second instruction name ; Use the data filling function, the fourth source operand, the fourth destination operand and the second instruction name to perform the data filling operation to obtain the data filling result.

Optionally, selecting the target operation function from the multiple candidate operation functions includes: selecting a floating-point scalar comparison function from the multiple candidate operation functions.

Optionally, using the target operation function and the target operation data to perform the target operation, and obtaining the target operation result includes: obtaining the target operation data, wherein the target operation data includes: a first floating-point number scalar and a second floating-point number scalar; using the floating-point number scalar The comparison function, the first floating-point scalar, and the second floating-point scalar perform a floating-point scalar comparison operation to obtain a floating-point scalar comparison result.

Optionally, using the floating-point scalar comparison function, the first floating-point scalar, and the second floating-point scalar to perform a floating-point scalar comparison operation, and obtaining a floating-point scalar comparison result includes: writing the first floating-point scalar into the first tensor, And write the second floating point scalar into the second tensor; use the vector subtraction instruction to obtain the difference operation result between the first tensor and the second tensor, and convert the difference operation result into an integer scalar; perform floating operation based on the integer scalar Point scalar comparison operation, get the result of floating-point scalar comparison.

Optionally, selecting the target operation function from the multiple candidate operation functions includes: selecting a division calculation function from the multiple candidate operation functions.

Optionally, using the target operation function and the target operation data to perform the target operation, and obtaining the target operation result includes: obtaining the target operation data, wherein the target operation data includes: the fifth source operand, the fifth destination operand and the second data amount. ; Use the division calculation function, the fifth source operand, the fifth destination operand and the second data amount to perform the division calculation operation to obtain the division calculation result.

According to one of the embodiments of the present application, a high-performance operator generation device suitable for Huawei Ascend chips is also provided, including: a generation module for generating a plurality of candidate operation functions in a target development mode, wherein the target The development method is based on the tensor iterator kernel development method determined by the tensor acceleration engine operator development framework of the Ascend AI processor; the selection module is used to select the target operation function to be used from multiple candidate operation functions; the processing module , which is used to perform the target operation using the target operation function and target operation data to obtain the target operation result.

Optionally, the selection module is further configured to select a data handling function from a plurality of candidate operation functions.

Optionally, the processing module is further configured to: obtain target operation data, wherein the target operation data includes: a first source operand, a first destination operand, and a data length; using a data transfer function, a first source operand, a first The destination operand and data length perform the data transfer operation to obtain the data transfer result.

Optionally, the processing module is also used to: use the data length to perform block processing on the first source operand to obtain a block result; when it is determined based on the block result that there is no tail block, the data transfer function and the block result are used. The first source operand is transferred to the first destination operand, and the data transfer result is obtained; when it is determined that there is a tail block based on the block result, the target transfer mode is determined according to the storage location of the first source operand and the first destination operand, and The first source operand is transferred to the first destination operand through the data transfer function and the target transfer method, and the data transfer result is obtained.

Optionally, the selection module is further configured to select a precision vector calculation function from a plurality of candidate operation functions.

Optionally, the processing module is further configured to: obtain target operation data, wherein the target operation data includes: the second source operand, the second destination operand, and the first instruction name; the calculation function using the precision vector, the second source operand , the second destination operand and the first instruction name to perform the precision vector calculation operation to obtain the precision vector calculation result.

Optionally, the processing module is further configured to: when the second source operand is not located in the global memory, determine the number of times of precision vector calculation, and use the precision vector calculation function and the first instruction name to perform a precision vector calculation operation on the second source operand to Obtain the calculation result of the precision vector, and transfer the calculation result of the precision vector to the second destination operand; when the second source operand is located in the global memory, perform multi-core optimization processing on the second source operand to obtain the optimization processing result, and determine the precision vector Calculate the number of times, use the precision vector calculation function and the first instruction name to perform the precision vector calculation operation on the optimization processing result to obtain the precision vector calculation result, and transfer the precision vector calculation result to the second destination operand.

Optionally, the selection module is further configured to select a reduction vector calculation function from a plurality of candidate operation functions.

Optionally, the processing module is further configured to: obtain target operation data, wherein the target operation data includes: a third source operand, a third destination operand, and a first data amount; using a reduction vector to calculate a function, a third source operand , the third destination operand and the first data amount to perform multiple rounds of iterative processing to obtain the reduction vector calculation result.

Optionally, the selection module is further configured to select a data filling function from a plurality of candidate operation functions.

Optionally, the processing module is further configured to: obtain target operation data, wherein the target operation data includes: a fourth source operand, a fourth destination operand, and a second instruction name; the data filling function, the fourth source operand, The fourth destination operand and the second instruction name perform a data filling operation to obtain a data filling result.

Optionally, the selection module is further configured to select a floating-point scalar comparison function from a plurality of candidate operation functions.

Optionally, the processing module is further configured to: obtain target operation data, wherein the target operation data includes: a first floating-point scalar and a second floating-point scalar; using a floating-point scalar comparison function, the first floating-point scalar and the second floating-point scalar; Point scalars perform a floating-point scalar comparison operation to obtain a floating-point scalar comparison result.

Optionally, the processing module is further configured to: write the first floating-point scalar into the first tensor, and write the second floating-point scalar into the second tensor; use the vector subtraction instruction to obtain the first tensor and the second tensor; The difference operation result of the quantity is converted into an integer scalar; the floating-point scalar comparison operation is performed based on the integer scalar, and the floating-point scalar comparison result is obtained.

Optionally, the selection module is further configured to select a division calculation function from a plurality of candidate operation functions.

Optionally, the processing module is further configured to: obtain target operation data, wherein the target operation data includes: the fifth source operand, the fifth destination operand, and the second data amount; the division calculation function, the fifth source operand, A division calculation operation is performed on the fifth destination operand and the second data amount to obtain a division calculation result.

According to one of the embodiments of the present application, a non-volatile storage medium is also provided, and a computer program is stored in the storage medium, wherein the computer program is configured to execute any one of the above-mentioned functions applicable to the Huawei Ascend chip when running. high-performance operator generation method.

According to one of the embodiments of the present application, there is also provided a processor for running a program, wherein the program is configured to execute any one of the above-mentioned high-performance operator generation methods applicable to Huawei Ascend chips when running. .

According to one of the embodiments of the present application, an electronic device is also provided, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the above-mentioned applications suitable for Huawei Ascend A high-performance operator generation method for chips.

In the embodiment of the present application, multiple candidate operation functions are generated in the target development mode, and the target development mode is the tensor iterator kernel development mode determined based on the tensor acceleration engine operator development framework of the Ascend artificial intelligence processor , and then select the target operation function to be used from multiple candidate operation functions, and finally use the target operation function and the target operation data to perform the target operation to obtain the target operation result, so as to quickly use the target operation function to perform the target operation on the target operation data. The purpose of the target operation result is to achieve the technical effect of improving the development efficiency of the operator and reducing the development workload, thereby solving the technical problem of low development efficiency of the high-performance operator in the related art.

Description of drawings

The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The schematic embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the attached image:

1 is a schematic diagram of an internal architecture of an AI Core according to the prior art;

2 is a hardware structural block diagram of a computer terminal for implementing a high-performance operator generation method suitable for Huawei Ascend chips according to an embodiment of the present application;

3 is a flowchart of a method for generating high-performance operators suitable for Huawei Ascend chips according to an embodiment of the present application;

4 is a schematic diagram of an intermediate result matrix according to an embodiment of the present application;

5 is a schematic diagram of yet another intermediate result matrix according to an embodiment of the present application;

FIG. 6 is a structural block diagram of a device for generating high-performance operators suitable for Huawei Ascend chips according to an embodiment of the present application.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only The embodiments are part of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the scope of protection of the present application.

It should be noted that the terms "first", "second", etc. in the description and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

First of all, some nouns or terms that appear in the process of describing the embodiments of the present application are suitable for the following explanations:

Ascend series processors are NPUs based on DaVinci architecture. Compared with traditional CPU/GPU chips, Ascend chips have higher performance and lower power in neural network training and inference. consumption.

DaVinci architecture is a new computing architecture for AI computing features. It has the characteristics of high computing power, high energy efficiency, flexibility and tailoring. It can efficiently perform related operations on vectors and tensors, and improve the efficiency of neural network operations. In order to improve the completeness of AI computing and the computing efficiency in different scenarios, a variety of computing units can be integrated into the DaVinci architecture.

The Ascend AI processor contains several AI computing cores (Core), which are responsible for executing vector- and tensor-related computationally intensive operators. The internal architecture of an AI Core is shown in the figure below. 1 is a schematic diagram of the internal architecture of an AI Core according to the prior art, wherein an AI Core includes: a system control module, an L1 buffer, an L1 input buffer controller, an L0A buffer, an L0C buffer, and an output buffer , matrix calculation unit (cube unit), vector calculation unit (vectorunit), scalar calculation unit (scalar unit), special register, general register, bus interface unit, scalar instruction processing queue, instruction sending module, event synchronization module, etc.

Specifically, the input buffer (L1 buffer) is a data transfer area in the AI Core for temporarily storing data that the AI Core needs to use repeatedly, thereby reducing the number of times the AI Core reads and writes these data from the bus. The output buffer (Unified Buffer, UB) is the data transfer area in the AI Core used to store the input and output data of the vector computing unit and the scalar computing unit. Before performing vector computing, the data needs to be moved to the output buffer before entering. The vector computing unit performs operations. Global Memory (GM), the external storage of AI Core, is used to store data waiting for AI calculation. When performing AI calculations, data must be moved to the interior of the AI Core to perform AI calculations.

The related scheme provides two development methods for operators: DSL and TIK. In the DSL development method, the developer only needs to call some highly encapsulated interfaces to express the calculation process, and then the compiler completes the subsequent automatic scheduling and operator code generation. The disadvantage of the DSL development method is that it is not flexible enough to develop some complex operators; the performance of the developed operators is generally poor. The TIK development method is more flexible, and more complex operators can be developed; TIK has better performance, and can be dozens or even thousands of times faster than DSL in many operators such as matrix multiplication and tensor slicing. However, the TIK development method requires a higher level of developers and is more time-consuming and labor-intensive.

When using the TIK development method for operator development, developers need to spend a lot of time and energy writing code related to data partitioning, data alignment, multi-core and multi-threading optimization, buffer management, etc. In the DSL development method, these operations are performed by automatically generated by the compiler. In order to achieve the same function, the workload of the TIK development method is usually more than ten times that of the DSL development method, so the development efficiency of using the TIK development method to develop operators is low.

The DSL development method can achieve good development efficiency, but the performance of the operators developed by the DSL development method is poor; the TIK development method can manually develop high-performance operators by developers, but the development efficiency is low. There is a technical problem in the related art that the development efficiency of high-performance operators is low.

According to one of the embodiments of the present application, an embodiment of a method for generating high-performance operators applicable to Huawei Ascend chips is provided. It should be noted that the steps shown in the flowcharts in the accompanying drawings may Executable instructions are executed in a computer system and, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that herein.

The method embodiments provided in the embodiments of the present application may be executed in a computer terminal or a similar computing device. FIG. 2 is a hardware structural block diagram of a computer terminal for implementing a method for generating high-performance operators suitable for Huawei Ascend chips according to an embodiment of the present application. As shown in FIG. 2, the computer terminal 20 may include one or more (202a, 202b, . processing means of a logic device FPGA or the like), a memory 204 for storing data, and a transmission means 206 for communication functions. In addition, may also include: display, input/output interface (I/O interface), universal serial bus (USB) port (may be included as one of the ports of the BUS bus), network interface, power supply and/or or camera. Those of ordinary skill in the art can understand that the structure shown in FIG. 2 is only for illustration, and does not limit the structure of the above electronic device. For example, the computer terminal 20 may also include more or fewer components than shown in FIG. 2 , or have a different configuration than that shown in FIG. 2 .

It should be noted that the one or more processors 202 and/or other data processing circuits described above may generally be referred to herein as "data processing circuits." The data processing circuit may be embodied in whole or in part as software, hardware, firmware or any other combination. Furthermore, the data processing circuitry may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in computer terminal 20. As referred to in the embodiments of the present application, the data processing circuit acts as a kind of processor control (eg, selection of a variable resistance termination path connected to an interface).

The memory 204 can be used to store software programs and modules of the application software, such as the program instruction/data storage device corresponding to the high-performance operator generation method applicable to the Huawei Ascend chip in the embodiment of the present application, the processor 202 is stored in the memory by running The software programs and modules in 204 are used to execute various functional applications and data processing, that is, to implement the above-mentioned high-performance operator generation method suitable for Huawei Ascend chips. Memory 204 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, memory 204 may further include memory located remotely from processor 202, which may be connected to computer terminal 20 through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

Transmission means 206 are used to receive or transmit data via a network. A specific example of the above-mentioned network may include a wireless network provided by a communication provider of the computer terminal 20 . In one example, the transmission device 206 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through the base station so as to communicate with the Internet. In one example, the transmission device 206 may be a radio frequency (Radio Frequency, RF) module, which is used for wirelessly communicating with the Internet.

The display may be, for example, a touch screen type liquid crystal display (LCD) that enables a user to interact with the user interface of the computer terminal 20 (or mobile device).

It should be noted here that, in some optional embodiments, the computer device shown in FIG. 2 may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or hardware elements A combination of both components and software components. It should be noted that FIG. 2 is only one example of a specific specific example, and is intended to illustrate the types of components that may be present in the computer apparatus described above.

Under the above operating environment, the present application provides a high-performance operator generation method suitable for Huawei Ascend chips as shown in FIG. 3 , which can be executed by the computer terminal shown in FIG. 2 or a similar computing device. FIG. 3 is a flowchart of a method for generating high-performance operators suitable for Huawei Ascend chips according to one of the embodiments of the present application. As shown in FIG. 3 , the process includes the following steps:

Step S31, under the target development mode, generate a plurality of candidate operation functions, wherein the target development mode is a tensor iterator kernel development mode determined based on the tensor acceleration engine operator development framework of the Ascend artificial intelligence processor;

The above-mentioned multiple candidate functions include: a data transfer function, a precision vector calculation function, a reduction vector calculation function, a data filling function, a floating-point scalar comparison function, and a division calculation function. Among them, different functions can operate on different operation data, and then obtain corresponding operation results.

Step S32, select the target operation function to be used from a plurality of candidate operation functions;

Specifically, for an implementation process of selecting a target operation function to be used from a plurality of candidate operation functions, reference may be made to the further introduction to the embodiments of the present application, which will not be repeated.

In step S33, the target operation is performed by using the target operation function and the target operation data, and the result of the target operation is obtained.

Optionally, different target operation functions can perform different target operations on the target operation data, thereby obtaining corresponding target operation results.

In an optional implementation manner, when the target operation function is a data transfer function, the data transfer operation is performed using the data transfer function and the target operation data to obtain a data transfer result.

In an optional implementation manner, when the target operation function is a precision vector calculation function, the precision vector calculation operation is performed by using the precision vector calculation function and the target operation data to obtain a precision vector calculation result.

In an optional embodiment, when the target operation function is a reduction vector calculation function. Using the reduction vector calculation function and the target operation data to perform multiple rounds of iterative processing, the reduction vector calculation result is obtained.

In an optional implementation manner, when the target operation function is a data filling function, a data filling operation is performed using the data filling function and the target operation data to obtain a data filling result.

In an optional implementation, when the target operation function is a floating-point scalar comparison function, a floating-point scalar comparison operation is performed using the floating-point scalar comparison function and the target operation data to obtain a floating-point scalar comparison result.

In an optional implementation manner, when the target operation function is a division calculation function, a division calculation operation is performed by using the division calculation function and the target operation data to obtain a division calculation result.

Specifically, for the implementation process of performing the target operation by using the target operation function and the target operation data, and obtaining the target operation result, reference may be made to the further introduction of the embodiments of the present application, which will not be repeated.

Through the above steps S31 to S33, multiple candidate operation functions are generated in the target development mode, and the target development mode is based on the tensor iterator kernel development determined by the tensor acceleration engine operator development framework of the Ascend AI processor method, and then select the target operation function to be used from multiple candidate operation functions, and finally use the target operation function and the target operation data to perform the target operation to obtain the target operation result. The purpose of obtaining the target operation result is to achieve the technical effect of improving the development efficiency of the operator and reducing the development workload, thereby solving the technical problem of low development efficiency of the high-performance operator in the related art.

The high-performance operator generation method applicable to the Huawei Ascend chip described in the above-mentioned embodiment will be further introduced below.

Optionally, in step S32, selecting a target operation function from a plurality of candidate operation functions includes:

Step S321, selecting a data transfer function from a plurality of candidate operation functions.

Specifically, the data handling function includes algorithms for automatically performing data partitioning and data transfer.

Optionally, in step S33, using the target operation function and the target operation data to perform the target operation, obtaining the target operation result includes:

Step S41, acquiring target operation data, wherein the target operation data includes: a first source operand, a first destination operand and a data length;

The above-mentioned first source operand includes the data block to be transported, and the data block to be transported in the first source operation data can be transported to the first destination operand by executing the data transport operation.

Step S42, using the data transfer function, the first source operand, the first destination operand and the data length to perform a data transfer operation to obtain a data transfer result.

Specifically, using the data transfer function, the first source operand, the first destination operand, and the data length to perform the data transfer operation, to obtain the data transfer result, reference may be made to the further introduction in the following embodiments, which will not be repeated.

Based on the above steps S41 to S42, the data handling function, the first source operand, the first destination operand and the data length can be used to perform the data handling operation, and the data handling result can be obtained quickly, thereby improving the data handling when developing high-performance operators. operational efficiency.

Optionally, in step S42, the data handling operation is performed using the data handling function, the first source operand, the first destination operand and the data length, and the obtained data handling result includes:

Step S421, using the data length to perform block processing on the first source operand to obtain a block result;

In the Ascend AI processor, the smallest unit of data transfer is a block, and one block is equal to 32 bytes. If the data length is an integer multiple of 32 bytes, the block result includes the data part that can be divided into whole blocks; if the data length is not an integer multiple of 32 bytes, the block result includes not only the data part that can be divided into whole blocks , and also includes the portion of data that cannot be divided into whole blocks, the tail block.

Step S422, when it is determined that there is no tail block based on the block result, the first source operand is transferred to the first destination operand by the data transfer function and the block result, and the data transfer result is obtained;

Step S423, when it is determined that there is a tail block based on the block result, the target handling method is determined according to the storage position of the first source operand and the first destination operand, and the first source operand is handled by the data handling function and the target handling method. To the first destination operand, the data transfer result is obtained.

Specifically, when it is determined that there is a tail block based on the block division result, it is determined that the storage locations of the first source operand and the first destination operand are located in the global memory or the buffer, and then the target transport mode is determined.

As an optional implementation manner, when both the first source operand and the first destination operand are located in the buffer, a move instruction (set_as) is used to move the tail block data of the first source operand to the first destination operation one by one. in number.

As an optional implementation manner, when the first source operand is located in the buffer and the first destination operand is located in the global memory, a buffer temporary storage space with a capacity of one block is requested, and the set_as instruction is used to manipulate the first source The data of the block at the end of the number is transferred to the temporary buffer space one by one, and then the block is transferred to the first destination operand at a time.

As an optional implementation, when the first source operand is located in the global memory, the first destination operand is in the buffer, and the buffer temporary storage space with a capacity of one block is applied for, and the end of the first source operand is The block is moved to the buffer temporary space at a time, and then the set_as instruction is used to move the data in the buffer temporary space to the first destination operand one by one.

As an optional implementation, when both the first source operand and the first destination operand are located in the global memory, apply for a buffer temporary storage space with a capacity divisible by 32 bytes, and calculate the number of data transfers that need to be performed , and then use the loop instruction (for_range) to transfer data. In the last cycle, the handling of the tail block needs to be considered, and the handling process at this time may refer to the implementation process of handling the tail block in the above-mentioned embodiment, and will not be repeated. If the amount of data is less than one block in the last loop, the last block of the penultimate loop will be transported together to ensure that there is at least one block of data.

Based on the above steps S421 to S423, the first source operand is subjected to block processing by using the data length to obtain a block result; when it is determined based on the block result that there is no tail block, the first source operand is processed by the data transfer function and the block result. A source operand is transferred to the first destination operand, and the data transfer result is obtained; when it is determined that there is a tail block based on the block result, the target transfer mode is determined according to the storage location of the first source operand and the first destination operand, and the The data transfer function and the target transfer method transfer the first source operand to the first destination operand, obtain the data transfer result, and can automatically perform data partitioning and data transfer between different storage locations. Compared with the related art, which cannot directly implement data transfer from global memory to global memory, and requires developers to manually transfer data through a buffer, the high-performance operator generation method suitable for Huawei Ascend chips in the embodiment of the present application can automatically Perform data partitioning and data transfer to effectively improve the data handling efficiency when developing high-performance operators.

Optionally, in step S32, selecting a target operation function from a plurality of candidate operation functions includes:

Step S322, selecting a precision vector calculation function from a plurality of candidate operation functions.

The above-mentioned precision vector calculation functions include general vector calculation and high-precision vector calculation. Among them, the general vector calculation includes the following six calculation types: monocular, binocular, scalar binocular, scalar trinocular, comparison/selection, and precision conversion. Compared with general vector calculations, high-precision vector calculations require additional buffer scratch space. In the related art, the developer needs to calculate the size of the temporary storage space. In the embodiment of the present application, the size of the temporary storage space can be automatically calculated by using the precision vector calculation function, which improves the calculation efficiency of the precision vector when developing operators.

Optionally, in step S33, using the target operation function and the target operation data to perform the target operation, obtaining the target operation result includes:

Step S51, obtaining target operation data, wherein the target operation data includes: the second source operand, the second destination operand and the first instruction name;

The above-mentioned second source operand includes the data to be calculated by the precision vector, and the result of the precision vector calculation can be transferred to the second destination operand. The first instruction includes a monocular vector calculation instruction, a binocular vector calculation instruction, and a scalar binocular vector calculation instruction. , one or more of scalar ternary vector calculation instructions, compare/select instructions, and precision conversion instructions.

Step S52, using the precision vector calculation function, the second source operand, the second destination operand and the first instruction name to perform the precision vector calculation operation to obtain the precision vector calculation result.

Specifically, use the precision vector calculation function, the second source operand, the second destination operand, and the first instruction name to perform the precision vector calculation operation, and obtain the implementation process of the precision vector calculation result with reference to the further introduction in the following embodiments. Repeat.

Based on the above steps S51 to S52, the precision vector calculation function, the second source operand, the second destination operand and the first instruction name can be used to perform the precision vector calculation operation, and the precision vector calculation result can be obtained quickly, thereby improving the development of high-performance computing. Efficiency of precision vector computations for subtimes.

Optionally, in step S52, using the precision vector calculation function, the second source operand, the second destination operand and the first instruction name to perform the precision vector calculation operation, and obtaining the precision vector calculation result includes:

Step S521, when the second source operand is not located in the global memory, determine the number of times of precision vector calculation, use the precision vector calculation function and the first instruction name to perform the precision vector calculation operation on the second source operand to obtain the precision vector calculation result, and Move the calculation result of the precision vector to the second destination operand;

Specifically, taking the first instruction as a monocular vector calculation instruction as an example, since the amount of data processed by the vector processing unit on the AI Core is limited each time, it is necessary to determine the number of calculations of the vector processing unit according to the data length on the AI Core, that is, Determines the number of precision vector calculations. After the number of precision vector calculations is determined, the second source operand is cyclically divided into blocks, the TIK instruction is called to perform the precision vector calculation operation, and the precision vector calculation result is sent to the second destination operand. During this period, multi-threading optimization is enabled to improve execution performance.

As an optional implementation, if the second destination operand is located in the buffer, the second destination operand is directly specified in the first instruction; if the second destination operand is not located in the buffer, the precision vector The calculation result is stored in the temporary storage space in the buffer, and then the data transfer function in the above embodiment is called to transfer the calculation result of the precision vector in the temporary storage space to the second destination operand.

Step S522, when the second source operand is located in the global memory, perform multi-core optimization processing on the second source operand to obtain the optimization processing result, determine the number of times of calculation of the precision vector, and use the precision vector calculation function and the first instruction name to optimize the processing result. The precision vector calculation operation is performed to obtain the precision vector calculation result, and the precision vector calculation result is transferred to the second destination operand.

Specifically, continuing to take the first instruction as a monocular vector calculation instruction as an example, when the second source operand is located in the global memory, multi-core optimization processing is performed on the second source operand to obtain the optimization processing result.

First, formulate a sub-core scheme: call the Application Programming Interface (API) of the TBE platform to obtain the number of AI Cores (CORE_NUM) of the current processor; let the data volume of the second source operand be n, and try to divide the n data into Divided into each AI Core. Since the smallest unit of data handling in the Ascend AI processor is a block, in order to avoid frequent tail block data processing, the data length n is converted into the number of blocks and rounded down to obtain the number of blocks (n_block). If n_block is equal to 0, it means that the amount of data is less than one block. In this case, there is no need to use multiple cores, and only a single AICore is used for processing. If n_block is greater than 0, the data of n_block blocks is evenly distributed to each AI Core. If it cannot be equally divided, the AI Core with a smaller serial number undertakes more processing tasks. For example, when n_block is equal to 34 and CORE_NUM is equal to 32, then let core 0 and core 1 process two blocks, and the remaining 30 cores each process one block.

Secondly, after the core division plan is formulated, the data is distributed to different AI Cores, the number of precision vector calculations is determined, and the precision vector calculation function is used and the first instruction name is used to perform the precision vector calculation operation on the optimized processing result to obtain the precision vector calculation. As a result, the precision vector calculation result is transferred to the second destination operand.

Based on the above steps S521 to S522, the precision vector calculation operation can be performed by using the precision vector calculation function, the second source operand, the second destination operand and the first instruction name, and the precision vector calculation result can be obtained quickly, which further improves the development of high-performance computing. Efficiency of precision vector computations for subtimes.

Optionally, in step S32, selecting a target operation function from a plurality of candidate operation functions includes:

In step S323, a reduction vector calculation function is selected from a plurality of candidate operation functions.

The functions of the above reduction vector calculation function include finding the maximum value and the index, finding the minimum value and the index, finding the sum of all values, and finding the product of all the values.

Since the reduction command in the TIK development method can only reduce 256 bytes of data at a time, for larger-scale data, developers need to manually perform multiple rounds of iterative processing. Using the high-performance operator generation method suitable for Huawei Ascend chips in this embodiment, the developer only needs to call the reduced vector calculation function and pass in the source operand, destination operand and data amount. In the reduced vector calculation function Multiple rounds of iterations will be performed automatically, which can improve the computational efficiency of reduced vectors when developing high-performance operators.

Optionally, in step S33, using the target operation function and the target operation data to perform the target operation, obtaining the target operation result includes:

Step S61, acquiring target operation data, wherein the target operation data includes: a third source operand, a third destination operand, and a first data amount;

The above-mentioned first data amount is the length of the third source operand.

Step S62, using the reduction vector calculation function, the third source operand, the third destination operand and the first data amount to perform multiple rounds of iterative processing to obtain a reduction vector calculation result.

Based on the above steps S61 to S62, the reduction vector calculation function, the third source operand, the third destination operand and the first data amount can be used to perform multiple rounds of iterative processing to quickly obtain the reduction vector calculation result, thereby improving the development of high-performance computing. Reduced vector computation efficiency for subtimes.

Specifically, taking the reduction vector calculation process of finding the maximum value and the index as an example, the above-mentioned multi-round iterative processing process is introduced.

First, calculate the number of rounds that need to be performed, and the length of the result produced by the first iteration. The result length is related to the length of the third source operand and the data type of the third source operand. For example, when the length of the third source operand is equal to 1024 and the data type is a 16-bit floating point type (float16), the vector processing unit can reduce 128 data of type float16 at a time and generate a result of length 2. Therefore, the resulting length of the first iteration is equal to 16. In the second round of calculation, the result of length 16 can be processed at one time, so the entire calculation process requires two rounds.

Secondly, according to the result of the iterative processing, an intermediate result matrix whose width is equal to the length of the result generated by the first iteration and whose height is equal to the number of calculation rounds is applied in the buffer. Next, several rounds of TIK reduction instructions are executed and the results are stored in the intermediate result matrix. Fig. 4 is a schematic diagram of an intermediate result matrix according to an embodiment of the present application. As shown in Fig. 4, n rounds of iterative processing are performed on the third source operand vector in the buffer, and the result of each round of iteration is stored in the intermediate result matrix in the corresponding row.

Further, the final index is calculated recursively bottom-up. In the results generated by the last iteration, the maximum value is the final maximum value, and the minimum value is the final minimum value. However, the obtained index is the index of the result generated by the penultimate iteration. Therefore, it is necessary to The final index is calculated recursively bottom-up.

For example, FIG. 5 is a schematic diagram of another intermediate result matrix according to an embodiment of the present application. As shown in FIG. 5 , the third source operand is [3, 13, 5, 9, 4, 8, 1, 6, 7 , 2, 14, 0, 11, 15, 10, 12], the vector computing unit can only reduce 4 numbers at a time, and the length of the first round of iterations is calculated to be equal to 8, which requires 3 rounds of iterations. Therefore, apply for a 3 × 8's intermediate result matrix.

In the first iteration, the first four digits of the source operand are reduced first: 13, 3, 5, 9; the maximum value is 13, which is at position 0, so 13 is filled in the first row of the intermediate result matrix , 0. Then look at the next four numbers: 4, 8, 1, 6; the maximum value is 8, which is in the first place, so. Fill in 8, 1 in the first row of the intermediate result matrix. By analogy, the first round of calculation is completed after four rounds of statutes.

In the second iteration, the first four numbers of the first row of the intermediate result matrix are first reduced: 13, 0, 8, 1. Among them, "0" and "1" belong to the index, so when the TIK instruction is called for reduction, it is shielded and does not participate in the actual reduction calculation. Finally, the result obtained in this round of iteration is: 13, 0, 15, 0; write the result to the second row of the intermediate result matrix.

In the third iteration, the results obtained are: 15, 2. Write the result to the third row of the intermediate matrix.

The final index is calculated recursively bottom-up. The index obtained in the last round, i.e. the third iteration, is equal to 2. According to the index 2, looking up the result of the second round of iteration gets "15, 0", that is, the index of the second round of iteration is equal to 0. According to index 0, look up the result of the first iteration to get "15, 2". According to index 2, the source operand is looked up, and the index in the source operand is obtained equal to 10. Therefore, the final result is equal to "15, 10", that is, the maximum value is equal to 15, and the index is equal to 10.

It should be noted that the implementation process of calculating the minimum value and index, calculating the sum of all values, and calculating the product of all values may refer to the above-mentioned implementation process of obtaining the maximum value, and will not be repeated.

When reducing the sum of all values and the product of all values, the result of a reduce instruction is not 2 numbers, but 1, compared to the process of finding the maximum value, because there is no index value; and there is no need to recursively Calculate the index value, and the result of the last iteration is the final result.

Optionally, in step S32, selecting a target operation function from a plurality of candidate operation functions includes:

Step S324, selecting a data filling function from a plurality of candidate operation functions.

Optionally, in step S33, using the target operation function and the target operation data to perform the target operation, obtaining the target operation result includes:

Step S71, acquiring target operation data, wherein the target operation data includes: a fourth source operand, a fourth destination operand, and a second instruction name;

Step S72, using the data filling function, the fourth source operand, the fourth destination operand and the second instruction name to perform the data filling operation to obtain a data filling result.

It should be noted that, the implementation process of using the data filling function, the fourth source operand, the fourth destination operand and the second instruction name to perform the data filling operation to obtain the data filling result can refer to the above-mentioned embodiment using the precision vector calculation function. , the second source operand, the second destination operand and the first instruction name to perform the precision vector calculation operation to obtain the realization process of the precision vector calculation result, which will not be repeated.

Based on the above steps S71 to S72, the data filling operation can be performed by using the data filling function, the fourth source operand, the fourth destination operand and the second instruction name, and the data filling result can be obtained quickly, thereby improving the development of high-performance operators. Data filling efficiency.

Optionally, in step S32, selecting a target operation function from a plurality of candidate operation functions includes:

Step S325, selecting a floating-point scalar comparison function from a plurality of candidate operation functions.

Optionally, in step S33, using the target operation function and the target operation data to perform the target operation, obtaining the target operation result includes:

Step S81, acquiring target operation data, wherein the target operation data includes: a first floating-point scalar and a second floating-point scalar;

Step S82, using the floating-point scalar comparison function, the first floating-point scalar, and the second floating-point scalar to perform a floating-point scalar comparison operation to obtain a floating-point scalar comparison result.

Specifically, use the floating-point scalar comparison function, the first floating-point scalar, and the second floating-point scalar to perform the floating-point scalar comparison operation, and obtain the implementation process of the floating-point scalar comparison result with reference to the further introduction in the following embodiments.

Based on the above steps S81 to S82, by using the floating-point scalar comparison function, the first floating-point scalar and the second floating-point scalar to perform the floating-point scalar comparison operation, the floating-point scalar comparison result can be obtained quickly, thereby improving the development of high-performance operators. Floating-point scalar comparison efficiency.

Optionally, in step S82, using the floating-point scalar comparison function, the first floating-point scalar, and the second floating-point scalar to perform a floating-point scalar comparison operation, and obtaining a floating-point scalar comparison result includes:

Step S821, writing the first floating-point scalar into the first tensor, and writing the second floating-point scalar into the second tensor;

Step S822, use the vector subtraction instruction to obtain the difference operation result between the first tensor and the second tensor, and convert the difference operation result into an integer scalar;

Step S823, performing a floating-point scalar comparison operation based on the integer scalar to obtain a floating-point scalar comparison result.

Specifically, the floating-point scalar comparison is converted into an integer scalar comparison by using floating-point number subtraction and data type static conversion.

For example, write two floating-point scalars (fp1 and fp2) that need to be compared into two tensors, that is, write fp1 to the first tensor and write fp2 to the second tensor.

Use the vector subtraction instruction to obtain the difference operation result between the first tensor and the second tensor (that is, fp1-fp2), and then write the operation result into a scalar and force it to be converted to an integer scalar. At this time, the contents of the two integer scalars are still is a floating point number, but is interpreted as an integer.

Determine the size of two floating-point numbers according to the positive or negative of the converted integer scalar. Since the binary representation of floating-point numbers and integers uses the highest bit as the sign bit, after the difference between the two floating-point numbers is converted into an integer scalar, the meaning of the sign bit is still preserved, and it can be judged from this that the original The size relationship between two floating-point numbers. The specific judgment process is as follows:

Determine whether fp1-fp2 is equal to 0, if it is equal, it means that fp1 is equal to fp2, and the algorithm ends; judge the highest bit of fp1-fp2, if it is changed to 0, it means that fp1 is greater than fp2, if the highest bit is equal to 1, it means that fp1 is less than fp2.

For example, to determine the size relationship between two single-precision floating-point numbers fp1 and fp2, where fp1 is equal to 3.7, its binary representation is "01000000011011001100110011001101". fp2 is equal to 4.1 and its binary representation is "01000000100000110011001100110011". fp1-fp2=-0.4, its binary representation is "10111110110011001100110011001000". The highest bit of the binary representation is equal to "1", indicating that the result of fp1-fp2 is a negative number, so it can be judged that fp1 is smaller than fp2.

Based on the above steps S821 to S823, by writing the first floating-point scalar into the first tensor, and writing the second floating-point scalar into the second tensor, the vector subtraction instruction is used to obtain the first tensor and the second tensor. The difference operation result of the tensor, and the difference operation result is converted into an integer scalar. Finally, the floating-point scalar comparison operation is performed based on the integer scalar to quickly obtain the floating-point scalar comparison result, which further improves the floating-point scalar when developing high-performance operators. Compare efficiency.

Optionally, in step S32, selecting a target operation function from a plurality of candidate operation functions includes:

Step S326, selecting a division calculation function from a plurality of candidate operation functions.

Optionally, in step S33, using the target operation function and the target operation data to perform the target operation, obtaining the target operation result includes:

Step S91, acquiring target operation data, wherein the target operation data includes: a fifth source operand, a fifth destination operand, and a second data amount;

Step S92, using the division calculation function, the fifth source operand, the fifth destination operand, and the second data amount to perform a division calculation operation to obtain a division calculation result.

Specifically, the precision vector calculation function is called, and the high precision vector calculation is used to perform the reciprocal operation and the multiplication operation to obtain the division calculation result. That is, first calculate the reciprocal of the divisor, and then multiply the reciprocal of the divisor with the dividend, so as to realize the division calculation operation and obtain the division calculation result.

Based on the above steps S91 to S92, the division calculation function, the fifth source operand, the fifth destination operand and the second data amount can be used to perform the division calculation operation, and the division calculation result can be obtained quickly, thereby improving the development of high-performance operators. Efficiency of division calculation.

The high-performance operator generation method applicable to the Huawei Ascend chip in the embodiment of this application is based on the TIK development method, and provides developers with a series of operation functions. These functions implement data handling, general vector calculation, high-precision vector calculation, and protocol Vector calculation, data filling, floating-point scalar comparison and division, and other operations, and automatic data block, data alignment, multi-core and multi-thread optimization, buffer management. The developer calls these operation functions to realize the operation logic of the operator, and the operation function replaces the developer to perform optimization and code generation, so that the developer does not need to care about details such as data partitioning, data alignment, multi-core and multi-threading optimization, buffer management, etc. The development time is shortened by more than 50%, thereby effectively improving the development efficiency in the process of developing high-performance operators.

In an optional embodiment, the algorithm in the embodiment of the present application is encapsulated into a TIK compiler to become a new TIK instruction, and a corresponding interface is provided to developers, thereby effectively improving development in the process of developing high-performance operators efficiency.

From the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in the various embodiments of this application.

In this embodiment, a high-performance operator generation device suitable for Huawei Ascend chips is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementations, and what has been described will not be repeated. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementations in hardware, or a combination of software and hardware, are also possible and contemplated.

FIG. 6 is a structural block diagram of a high-performance operator generation device suitable for Huawei Ascend chips according to an embodiment of the present application. As shown in FIG. 6 , the high-performance operator generation device 600 suitable for Huawei Ascend chips includes: :

The generating module 601 is used to generate a plurality of candidate operation functions under the target development mode, wherein the target development mode is the tensor iterator kernel development mode determined based on the tensor acceleration engine operator development framework of the Ascend artificial intelligence processor ;

A selection module 602 is used to select a target operation function to be used from a plurality of candidate operation functions;

The processing module 603 is configured to perform the target operation by using the target operation function and the target operation data to obtain the target operation result.

Optionally, the selection module 602 is further configured to select a data handling function from a plurality of candidate operation functions.

Optionally, the processing module 603 is further configured to: obtain target operation data, wherein the target operation data includes: the first source operand, the first destination operand and the data length; the data transfer function, the first source operand, the first source operand, the first Perform a data transfer operation with a destination operand and data length to obtain the data transfer result.

Optionally, the processing module 603 is further configured to: use the data length to perform block processing on the first source operand to obtain a block result; when it is determined based on the block result that there is no tail block, pass the data handling function and the block result. The first source operand is transported to the first destination operand, and the data handling result is obtained; when it is determined that there is a tail block based on the block result, the target handling mode is determined according to the storage location of the first source operand and the first destination operand, The first source operand is transferred to the first destination operand through the data transfer function and the target transfer method to obtain a data transfer result.

Optionally, the selection module 602 is further configured to select a precision vector calculation function from a plurality of candidate operation functions.

Optionally, the processing module 603 is further configured to: acquire target operation data, wherein the target operation data includes: the second source operand, the second destination operand and the first instruction name; the calculation function using the precision vector, the second source operation The number, the second destination operand and the name of the first instruction execute the precision vector calculation operation to obtain the precision vector calculation result.

Optionally, the processing module 603 is further configured to: when the second source operand is not located in the global memory, determine the number of times of precision vector calculation, and use the precision vector calculation function and the first instruction name to perform a precision vector calculation operation on the second source operand. To obtain the calculation result of the precision vector, and transfer the calculation result of the precision vector to the second destination operand; when the second source operand is located in the global memory, perform multi-core optimization processing on the second source operand to obtain the optimization processing result and determine the accuracy For the number of vector calculations, use the precision vector calculation function and the first instruction name to perform the precision vector calculation operation on the optimization processing result to obtain the precision vector calculation result, and transfer the precision vector calculation result to the second destination operand.

Optionally, the selection module 602 is further configured to select a reduction vector calculation function from a plurality of candidate operation functions.

Optionally, the processing module 603 is further configured to: obtain target operation data, wherein the target operation data includes: a third source operand, a third destination operand, and a first data amount; using a reduction vector to calculate a function, a third source operation Perform multiple rounds of iterative processing on the number, the third destination operand and the first data amount to obtain the reduction vector calculation result.

Optionally, the selection module 602 is further configured to select a data filling function from a plurality of candidate operation functions.

Optionally, the processing module 603 is further configured to: obtain target operation data, wherein the target operation data includes: the fourth source operand, the fourth destination operand and the second instruction name; the data filling function, the fourth source operand , the fourth destination operand and the second instruction name to perform a data filling operation to obtain a data filling result.

Optionally, the selection module 602 is further configured to select a floating-point scalar comparison function from a plurality of candidate operation functions.

Optionally, the processing module 603 is further configured to: acquire target operation data, wherein the target operation data includes: a first floating-point scalar and a second floating-point scalar; using a floating-point scalar comparison function, the first floating-point scalar and the second floating-point scalar The floating-point scalar performs a floating-point scalar comparison operation to obtain the floating-point scalar comparison result.

Optionally, the processing module 603 is further configured to: write the first floating-point scalar into the first tensor, and write the second floating-point scalar into the second tensor; use a vector subtraction instruction to obtain the first tensor and the second tensor. The difference operation result of the tensor is converted into an integer scalar; the floating-point scalar comparison operation is performed based on the integer scalar, and the floating-point scalar comparison result is obtained.

Optionally, the selection module 602 is further configured to select a division calculation function from a plurality of candidate operation functions.

Optionally, the processing module 603 is further configured to: obtain target operation data, wherein the target operation data includes: the fifth source operand, the fifth destination operand, and the second data amount; using the division calculation function, the fifth source operand , the fifth destination operand and the second data amount to perform a division calculation operation to obtain a division calculation result.

It should be noted that the above modules can be implemented by software or hardware, and the latter can be implemented in the following ways, but not limited to this: the above modules are all located in the same processor; or, the above modules can be combined in any combination The forms are located in different processors.

Embodiments of the present application further provide a non-volatile storage medium, where a computer program is stored in the non-volatile storage medium, wherein the computer program is configured to execute the steps in any of the above method embodiments when running .

Optionally, in this embodiment, the above-mentioned non-volatile storage medium may be configured to store a computer program for executing the following steps:

S1, in the target development mode, generate multiple candidate operation functions, wherein the target development mode is the tensor iterator kernel development mode determined based on the tensor acceleration engine operator development framework of the Ascend artificial intelligence processor;

S2, select the target operation function to be used from multiple candidate operation functions;

S3, use the target operation function and the target operation data to perform the target operation to obtain the target operation result.

Optionally, in this embodiment, the above-mentioned non-volatile storage medium may include but is not limited to: a U disk, a read-only memory (Read-Only Memory, referred to as ROM), and a random access memory (Random Access Memory, referred to as ROM) Various media that can store computer programs, such as RAM), mobile hard disks, magnetic disks or optical disks.

Embodiments of the present application also provide an electronic device, including a memory, a processor, a first wireless network chip and a second wireless network chip, wherein a computer program is stored in the memory, and the processor is configured to run a computer A program to perform the steps in any one of the above method embodiments.

Optionally, in this embodiment, the above-mentioned processor may be configured to execute the following steps through a computer program:

S1, in the target development mode, generate multiple candidate operation functions, wherein the target development mode is the tensor iterator kernel development mode determined based on the tensor acceleration engine operator development framework of the Ascend artificial intelligence processor;

S2, select the target operation function to be used from multiple candidate operation functions;

S3, use the target operation function and the target operation data to perform the target operation to obtain the target operation result.

Optionally, for specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and optional implementation manners, and details are not described herein again in this embodiment.

The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are only illustrative, for example, the division of the units may be a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components may be combined or Integration into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes .

The above are only the preferred embodiments of the present application. It should be pointed out that for those skilled in the art, without departing from the principles of the present application, several improvements and modifications can also be made. It should be regarded as the protection scope of this application.

Claims (20)

1. A method for generating high performance operators for Huaji Shengji chip, comprising:

generating a plurality of candidate operation functions in a target development mode, wherein the target development mode is a tensor iterator kernel development mode determined by a tensor acceleration engine operator development framework based on the rising artificial intelligence processor;

selecting a target operation function to be used from the plurality of candidate operation functions;

and executing target operation by using the target operation function and the target operation data to obtain a target operation result.

2. The method of claim 1, wherein selecting the target operation function from the plurality of candidate operation functions comprises:

a data-handling function is selected from the plurality of candidate operation functions.

3. The method as claimed in claim 2, wherein the performing a target operation using the target operation function and the target operation data to obtain the target operation result comprises:

obtaining the target operation data, wherein the target operation data comprises: a first source operand, a first destination operand, and a data length;

and executing data carrying operation by using the data carrying function, the first source operand, the first destination operand and the data length to obtain a data carrying result.

4. The method as claimed in claim 3, wherein the performing data-handling operations using the data-handling function, the first source operand, the first destination operand, and the data length to obtain the data-handling result comprises:

carrying out blocking processing on the first source operand by using the data length to obtain a blocking result;

when it is determined that no tail block exists based on the blocking result, carrying the first source operand to the first destination operand through the data carrying function and the blocking result to obtain the data carrying result;

and when the tail block is determined to exist based on the blocking result, determining a target carrying mode according to the storage positions of the first source operand and the first destination operand, and carrying the first source operand to the first destination operand through the data carrying function and the target carrying mode to obtain the data carrying result.

5. The method of claim 1, wherein selecting the target operation function from the plurality of candidate operation functions comprises:

a precision vector calculation function is selected from the plurality of candidate operation functions.

6. The method as claimed in claim 5, wherein the performing the target operation using the target operation function and the target operation data to obtain the target operation result comprises:

obtaining the target operation data, wherein the target operation data comprises: a second source operand, a second destination operand, and a first instruction name;

and executing precision vector calculation operation by using the precision vector calculation function, the second source operand, the second destination operand and the first instruction name to obtain a precision vector calculation result.

7. The method of claim 6, wherein performing a precision vector calculation operation using the precision vector calculation function, the second source operand, the second destination operand, and the first instruction name to obtain the precision vector calculation result comprises:

when the second source operand is not located in the global memory, determining the number of precision vector calculation, performing precision vector calculation operation on the second source operand by using the precision vector calculation function and the first instruction name to obtain a precision vector calculation result, and carrying the precision vector calculation result to the second destination operand;

when the second source operand is located in the global memory, performing multi-core optimization processing on the second source operand to obtain an optimization processing result, determining the precision vector calculation times, performing precision vector calculation operation on the optimization processing result by using the precision vector calculation function and the first instruction name to obtain the precision vector calculation result, and carrying the precision vector calculation result to the second destination operand.

8. The method of claim 1, wherein selecting the target operation function from the plurality of candidate operation functions comprises:

selecting a reduced vector calculation function from the plurality of candidate operation functions.

9. The method as claimed in claim 8, wherein the performing a target operation using the target operation function and the target operation data to obtain the target operation result comprises:

obtaining the target operation data, wherein the target operation data comprises: a third source operand, a third destination operand, and a first data volume;

and executing multiple rounds of iterative processing by using the reduced vector calculation function, the third source operand, the third destination operand and the first data volume to obtain a reduced vector calculation result.

10. The method of claim 1, wherein selecting the target operation function from the plurality of candidate operation functions comprises:

a data fill function is selected from the plurality of candidate operation functions.

11. The method as claimed in claim 10, wherein the performing a target operation using the target operation function and the target operation data to obtain the target operation result comprises:

obtaining the target operation data, wherein the target operation data comprises: a fourth source operand, a fourth destination operand, and a second instruction name;

and executing data filling operation by using the data filling function, the fourth source operand, the fourth destination operand and the second instruction name to obtain a data filling result.

12. The method of claim 1, wherein selecting the target operation function from the plurality of candidate operation functions comprises:

selecting a floating point number scalar compare function from the plurality of candidate operation functions.

13. The method as claimed in claim 12, wherein the performing a target operation using the target operation function and the target operation data to obtain the target operation result comprises:

obtaining the target operation data, wherein the target operation data comprises: a first floating point scalar and a second floating point scalar;

and executing a floating point scalar comparison operation by using the floating point scalar comparison function, the first floating point scalar and the second floating point scalar to obtain a floating point scalar comparison result.

14. The method of claim 13, wherein performing a floating point scalar compare operation using the floating point scalar compare function, the first floating point scalar and the second floating point scalar to obtain the floating point scalar compare result comprises:

writing the first floating point scalar to a first tensor, and writing the second floating point scalar to a second tensor;

obtaining a difference operation result of the first tensor and the second tensor by using a vector subtraction instruction, and converting the difference operation result into an integer scalar;

and executing a floating point scalar comparison operation based on the integer scalar to obtain a floating point scalar comparison result.

15. The method of claim 1, wherein selecting the target operation function from the plurality of candidate operation functions comprises:

a division calculation function is selected from the plurality of candidate operation functions.

16. The method as claimed in claim 15, wherein the performing a target operation using the target operation function and the target operation data to obtain the target operation result comprises:

obtaining the target operation data, wherein the target operation data comprises: a fifth source operand, a fifth destination operand, and a second data volume;

and executing division calculation operation by using the division calculation function, the fifth source operand, the fifth destination operand and the second data volume to obtain a division calculation result.

17. A high performance operator generator for Huaji Shengji chip, comprising:

a generation module for generating a plurality of candidate operation functions in a target development mode, wherein the target development mode is a tensor iterator kernel development mode determined by a tensor acceleration engine operator development framework based on the Itanium artificial intelligence processor;

a selecting module for selecting a target operation function to be used from the plurality of candidate operation functions;

and the processing module is used for executing the target operation by utilizing the target operation function and the target operation data to obtain a target operation result.

18. A non-volatile storage medium, wherein a computer program is stored in the storage medium, and wherein the computer program is configured to execute the method for generating a high performance operator for use in hua tiao chip of any one of claims 1 to 17 when running.

19. A processor for running a program, wherein the program is configured to execute the method for generating a high performance operator suitable for use in hua tiao chips of any one of claims 1 to 17.

20. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform the high performance operator generation method for Huaqi Shengji chip as claimed in any one of claims 1 to 17.

Images