WO2025086399A1 - Processing method for matrix multiplication in parallel computing hardware, and related device - Google Patents

Abstract

Embodiments of the present application provide a processing method for matrix multiplication in parallel computing hardware, and a related device. The processing method comprises: acquiring a first initial matrix and a second initial matrix; performing half-precision processing on the basis of a single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; obtaining a first difference matrix on the basis of the difference of the first initial matrix and the first half-precision matrix, and obtaining a second difference matrix on the basis of the difference of the second initial matrix and the second half-precision matrix; and finally, accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix. Thus, a single-precision multiplication result with relatively high precision is obtained on a hardware device that only supports half-precision multiplication.

Description

Processing method and related equipment for matrix multiplication operation in parallel computing hardware Technical Field

The present application relates to the field of artificial intelligence technology, and in particular to a processing method and related equipment for matrix multiplication operations in parallel computing hardware.

Background Art

In recent years, with the vigorous development of machine learning, artificial intelligence and other fields, the number of matrix multiplication operations has greatly increased, and the development of dedicated hardware for parallel computing has also greatly accelerated. As a result, improving the accuracy of matrix multiplication operations has become the most important thing to pay attention to.

In the related art, for the multiplication operation of single-precision matrices, a computing chip that supports single-precision calculation can be used for calculation, but the cost of the computing chip that supports single-precision calculation is relatively high. In addition, when using a computing chip that only supports half-precision calculation, the single-precision matrix to be multiplied is usually converted into a half-precision matrix, and then the half-precision is multiplied, but the accuracy of the result obtained by this solution is low.

Summary of the invention

The embodiments of the present application provide a method and related equipment for processing matrix multiplication operations in parallel computing hardware, which can improve the accuracy of single-precision matrix multiplication operations when using a computing chip that supports half-precision calculations.

To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for processing matrix multiplication operations in parallel computing hardware, comprising:

Obtain a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices;

Perform half-precision processing based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix;

Obtaining a first difference matrix based on a difference between the first initial matrix and the first half-precision matrix, and obtaining a second difference matrix based on a difference between the second initial matrix and the second half-precision matrix; wherein both the first difference matrix and the second difference matrix are half-precision matrices;

Accumulate the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and use the first single-precision target matrix as a result of matrix multiplication of the first initial matrix and the second initial matrix.

In some embodiments, obtaining a first difference matrix based on the difference between the first initial matrix and the first half-precision matrix includes:

Performing single-precision processing on the first half-precision matrix to obtain a first intermediate matrix;

Perform half-precision processing on the difference between the first initial matrix and the first intermediate matrix to obtain the first difference matrix.

In some embodiments, the accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix includes:

Accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first addition term;

Obtaining a second addition term according to the product of the first difference matrix and the second difference matrix;

The first addition item and the second addition item are accumulated to obtain the first single-precision target matrix.

In some embodiments, the elements in the first half-precision matrix include mantissas; the accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix further includes:

Obtaining a preset multiplication value determined according to the number of digits of the mantissa;

Multiplying the first difference matrix by a preset multiplication value to obtain a first magnification matrix, and multiplying the second difference matrix by the preset multiplication value to obtain a second magnification matrix;

Accumulate the product of the first half-precision matrix and the second magnification matrix, and the product of the second half-precision matrix and the first magnification matrix to obtain an intermediate magnification matrix;

Dividing the intermediate magnification matrix by the preset multiplication value to obtain an intermediate reduction matrix;

The product of the first half-precision matrix and the second half-precision matrix and the intermediate reduction matrix are accumulated to obtain the first single-precision target matrix.

In some embodiments, obtaining the first initial matrix and the second initial matrix includes:

Get a first target matrix and a second target matrix;

Obtaining a maximum matrix multiplication operation order of the parallel computing hardware;

Based on the maximum matrix multiplication operation order, the first target matrix is divided to obtain at least one first initial matrix, and the second target matrix is divided to obtain at least one second initial matrix.

In some embodiments, when there are multiple first initial matrix and second initial matrix, the method further includes:

Selecting the second initial matrix in the second target matrix according to the block position of the first initial matrix in the first target matrix, and generating a plurality of matrix sequences; each of the matrix sequences includes a plurality of the matrix groups, and each of the matrix groups includes the first initial matrix and the second initial matrix;

Accumulating the product of the first half-precision matrix and the second half-precision matrix in each of the matrix groups to obtain a first accumulated matrix of the matrix sequence, and accumulating the sum of the product of the first half-precision matrix and the second difference matrix and the product of the second half-precision matrix and the second difference matrix in each of the matrix groups to obtain a second accumulated matrix of the matrix sequence;

A second single-precision target matrix is obtained according to the first accumulation matrix and the second accumulation matrix, and the second single-precision target matrix is used as a result of matrix multiplication operation of the first target matrix and the second target matrix.

In some embodiments, the method further comprises:

Perform double-precision processing based on the single-precision data type to obtain a first double-precision matrix of the first target matrix, a second double-precision matrix of the second target matrix, and a check matrix of the second single-precision target matrix;

Performing a multiplication operation on the first double precision matrix and the second double precision matrix to obtain an evaluation matrix;

Obtaining a test result based on the test matrix and the evaluation matrix;

The test result is compared with a preset test threshold, and the second single-precision target matrix is output based on the comparison result.

To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a matrix multiplication operation device, comprising:

An acquisition module, used to acquire a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices;

A half-precision conversion module, used for performing half-precision processing based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix;

A difference processing module, configured to obtain a first difference matrix based on a difference between the first initial matrix and the first half-precision matrix, and to obtain a second difference matrix based on a difference between the second initial matrix and the second half-precision matrix; wherein both the first difference matrix and the second difference matrix are half-precision matrices;

A calculation module is used to accumulate the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and use the first single-precision target matrix as a result of matrix multiplication of the first initial matrix and the second initial matrix.

To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, wherein the electronic device includes a storage A memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the processing method for matrix multiplication operation in parallel computing hardware as described in the first aspect is implemented.

To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, it implements the processing method of matrix multiplication operations in the parallel computing hardware described in the first aspect above.

The processing method and related equipment of matrix multiplication operation in parallel computing hardware proposed in the embodiment of the present application are as follows: obtaining a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices; then performing half-precision processing based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; then obtaining a first difference matrix based on the difference between the first initial matrix and the first half-precision matrix, and obtaining a second difference matrix based on the difference between the second initial matrix and the second half-precision matrix; wherein the first difference matrix and the second difference matrix are both half-precision matrices; finally, accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and using the first single-precision target matrix as the result of matrix multiplication operation of the first initial matrix and the second initial matrix. The embodiments of the present application are directed to performing single-precision matrix multiplication operations on a device that supports half-precision matrix multiplication operations, using a first difference matrix to save the error after a first initial matrix is converted to a first half-precision matrix, and using a second difference matrix to save the error after a second initial matrix is converted to a second half-precision matrix, thereby adding an error compensation term in the multiplication operation of the first half-precision matrix and the second half-precision matrix to perform corresponding half-precision multiplication operations, and thereby obtaining a single-precision multiplication result with higher accuracy on a hardware device that only supports half-precision multiplication operations.

Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flowchart of a method for processing matrix multiplication operations in parallel computing hardware provided in an embodiment of the present application.

FIG. 2 is a flowchart of step S101 in FIG. 1 .

FIG3 is a schematic diagram of the structure of matrix blocks provided in an embodiment of the present application.

FIG. 4 is a schematic diagram of a floating-point data structure provided in an embodiment of the present application.

FIG. 5 is a flowchart of step S103 in FIG. 1 .

FIG. 6 is a flowchart of step S104 in FIG. 1 .

FIG. 7 is another flow chart of step S104 in FIG. 1 .

FIG8 is a flowchart of a matrix block multiplication operation provided in an embodiment of the present application.

FIG. 9 is another flowchart of a matrix block multiplication operation provided by an embodiment of the present application.

FIG. 10 is another flow chart of a method for processing matrix multiplication operations in parallel computing hardware provided in an embodiment of the present application.

FIG. 11 is a schematic diagram of an improved flow chart of a matrix block multiplication operation provided in an embodiment of the present application.

FIG. 12 is a schematic diagram of another improved flow chart of the matrix block multiplication operation provided in an embodiment of the present application.

FIG. 13 is another flowchart of a method for processing matrix multiplication operations in parallel computing hardware provided in an embodiment of the present application.

FIG14 is a schematic diagram of the chip structure of the Ascend AI processor provided in one embodiment of the present application.

FIG. 15 is a schematic diagram of the execution of block parallel computing provided in an embodiment of the present application.

FIG. 16 is a simulation diagram comparing the accuracy of a method for processing matrix multiplication operations in parallel computing hardware provided in an embodiment of the present application.

FIG. 17 is a simulation diagram comparing the working efficiency of a method for processing matrix multiplication operations in parallel computing hardware provided in an embodiment of the present application.

FIG. 18 is a simulation data table diagram of the working efficiency of a method for processing matrix multiplication operations in parallel computing hardware provided in an embodiment of the present application.

FIG. 19 is another accuracy comparison simulation diagram of a method for processing matrix multiplication operations in parallel computing hardware provided in an embodiment of the present application.

FIG. 20 is a schematic diagram of the structure of a matrix multiplication operation device provided in one embodiment of the present application.

FIG. 21 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

First, some nouns involved in this application are analyzed:

Single-precision matrix multiplication refers to the operation of multiplying two matrices of single-precision floating-point type. Single-precision floating-point type usually uses 32-bit binary numbers to represent a floating-point number, of which 1 bit is used for the sign bit, 8 bits are used for the exponent, and 23 bits are used for the mantissa. Assuming that the size of matrix A is m×n and the size of matrix B is n×p, then the size of their product C is m×p, where the value of each element C[i][j] of matrix C is the sum of the products of the elements in the i-th row of matrix A and the elements in the j-th column of matrix B.

Similar to single-precision matrix multiplication, half-precision matrix multiplication refers to the multiplication of two matrices of half-precision floating-point type. Half-precision floating-point type usually uses 16 bits of binary to represent a floating-point number, of which 1 bit is used for the sign bit, 5 bits for the exponent, and 10 bits for the mantissa. Since the precision of half-precision floating-point type is lower, in practical applications, half-precision matrix multiplication is usually used in scenarios that require less precision, such as calculations in neural networks.

IEEE-754 is a floating-point representation of binary numbers. It is an international standard developed by the Institute of Electrical and Electronics Engineers (IEEE). This standard specifies the binary representation of floating-point numbers in computers, including the sign bit, exponent bit, and mantissa bit.

Parallel computing hardware refers to a hardware device that can perform multiple computing tasks simultaneously. These hardware devices usually have a high degree of parallel computing capabilities and efficient computing resource management capabilities, which can achieve efficient computing and data processing.

In recent years, with the vigorous development of machine learning, artificial intelligence and other fields, the number of matrix multiplication operations has greatly increased, and the development of dedicated hardware for parallel computing has also greatly accelerated. As a result, improving the accuracy of matrix multiplication operations has become the most important thing to pay attention to.

In the related art, for the multiplication operation of single-precision matrices, a computing chip that supports single-precision calculation can be used for calculation, but the cost of the computing chip that supports single-precision calculation is relatively high. In addition, when using a computing chip that only supports half-precision calculation, the single-precision matrix to be multiplied is usually converted into a half-precision matrix, and then the half-precision is multiplied, but the accuracy of the result obtained by this solution is low.

Based on this, the embodiment of the present application provides a processing method and related equipment for matrix multiplication operations in parallel computing hardware, which can improve the accuracy of single-precision matrix multiplication operations in computing chips that support half-precision calculations. The processing method for matrix multiplication operations in parallel computing hardware mainly obtains a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices; then half-precision processing is performed based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; then a first difference matrix is obtained based on the difference between the first initial matrix and the first half-precision matrix, and a second difference matrix is obtained based on the difference between the second initial matrix and the second half-precision matrix; wherein the first difference matrix and the second difference matrix are both half-precision matrices; finally, the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix are accumulated to obtain a first single-precision target matrix, and the first single-precision target matrix is used as the result of matrix multiplication operations of the first initial matrix and the second initial matrix. The embodiment of the present application is directed to performing a single-precision matrix multiplication operation on a device supporting a half-precision matrix multiplication operation, using a first difference matrix to save a first initial matrix converted to The error after the first half-precision matrix is obtained, and the error after the second initial matrix is converted to the second half-precision matrix is saved by using the second difference matrix, so as to add an error compensation term in the multiplication operation of the first half-precision matrix and the second half-precision matrix to perform the corresponding half-precision multiplication operation, and then obtain a single-precision multiplication operation result with higher accuracy on a hardware device that only supports half-precision multiplication operation.

The embodiments of the present application provide a processing method and related equipment for matrix multiplication operations in parallel computing hardware, which are specifically illustrated by the following embodiments. First, the processing method for matrix multiplication operations in parallel computing hardware in the embodiments of the present application is described.

The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

The processing method of matrix multiplication operation in parallel computing hardware provided by the embodiment of the present application relates to the field of artificial intelligence technology, and in particular to the field of data calculation and processing. The processing method of matrix multiplication operation in parallel computing hardware provided by the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be a computer program running in a terminal or a server side. For example, a computer program can be a native program or a software module in an operating system; it can be a local application (Application, APP), that is, a program that needs to be installed in an operating system to run, or it can be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be an application, module or plug-in in any form. Among them, the terminal communicates with the server via a network. The processing method of matrix multiplication operation in the parallel computing hardware can be executed by a terminal or a server, or by a terminal and a server in collaboration.

In some embodiments, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc. In addition, the terminal may also be an intelligent vehicle-mounted device. The intelligent vehicle-mounted device applies the processing method of matrix multiplication operation in the parallel computing hardware of this embodiment to provide related services and improve the driving experience. The server may be an independent server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms; or a service node in a blockchain system, where each service node in the blockchain system forms a peer-to-peer (Peer To Peer, P2P) network, and the P2P protocol is an application layer protocol running on the Transmission Control Protocol (Transmission Control Protocol, TCP) protocol. The server can be installed with a server end of a text translation system, through which the server end can interact with the terminal, for example, the corresponding software is installed on the server end, and the software may be an application that implements the processing method of matrix multiplication operation in parallel computing hardware, etc., but is not limited to the above forms. The terminal and the server can be connected via Bluetooth, Universal Serial Bus (USB) or network and other communication connection methods, which is not limited in this embodiment.

The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network personal computers (Personal Computer, PC), minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

In some embodiments, the processing method of matrix multiplication operation in parallel computing hardware provided in the present application is mainly aimed at the matrix multiplication operation of two single-precision matrices in parallel computing hardware that only supports half-precision matrix multiplication operation. The parallel computing hardware that only supports half-precision matrix multiplication operation can be Ascend AI chip, etc.

First, a method for processing matrix multiplication operations in parallel computing hardware in an embodiment of the present application is described. In this embodiment, The processing method of matrix multiplication operation in parallel computing hardware can be applied to a matrix multiplication operation device. Referring to FIG1 , an optional flowchart of the processing method of matrix multiplication operation in parallel computing hardware provided in an embodiment of the present application is provided. The method in FIG1 may include but is not limited to steps S101 to S104. At the same time, it can be understood that this embodiment does not specifically limit the order of steps S101 to S104 in FIG1 , and the order of steps can be adjusted or certain steps can be reduced or increased according to actual needs.

Step S101: Obtain a first initial matrix and a second initial matrix.

In some embodiments, after the matrix multiplication operation device responds to the single-precision matrix multiplication operation, it first obtains the first initial matrix and the second initial matrix, both of which are single-precision matrices. The first initial matrix and the second initial matrix refer to matrix data to be processed by the matrix multiplication operation. In this embodiment, there is no restriction on the acquisition source of the first initial matrix and the second initial matrix, that is, they can be input manually, or generated based on the calculation of the machine learning model itself, or extracted from a text database by a computer device, or crawled from the network by a computer device, etc.

In some embodiments, the first initial matrix is derived from the first target matrix, the second initial matrix is derived from the second target matrix, and the first target matrix and the second target matrix are two matrices that need to be matrix multiplied. In some cases, the data scale of the first target matrix and the second target matrix is large, such as the input matrix scale of matrix multiplication operations in some applications of network models in deep learning is generally large. Therefore, in order to improve the operational efficiency of the matrix multiplication operation device, and also limited by the running memory size of the parallel computing hardware, it is necessary to perform block processing on the first target matrix and the second target matrix. The following describes the process of performing block processing on the first target matrix and the second target matrix in an embodiment of the present application.

Therefore, referring to FIG. 2 , obtaining the first initial matrix and the second initial matrix includes steps S201 to S203 .

Step S201: Obtain a first target matrix and a second target matrix.

Step S202: Obtain the maximum matrix multiplication operation order of the parallel computing hardware.

Step S203: Based on the maximum matrix multiplication operation order, the first target matrix is divided to obtain at least one first initial matrix, and the second target matrix is divided to obtain at least one second initial matrix.

In some embodiments, in some embodiments, after responding to the single-precision matrix multiplication operation, the matrix multiplication operation device first obtains a first target matrix and a second target matrix, both of which are single-precision matrices, and then obtains the maximum matrix multiplication operation order in the parallel computing hardware. It can be understood that the maximum matrix multiplication operation order can be obtained through the running memory size of the parallel computing hardware.

In some embodiments, a judgment is made based on the data size of the first target matrix and the second target matrix and the maximum matrix multiplication operation order in the parallel computing hardware. If the data size of the first target matrix and/or the second target matrix is greater than the maximum matrix multiplication operation order in the parallel computing hardware, the first target matrix and the second target matrix are processed in blocks as follows.

3 is a schematic diagram of a matrix block structure provided by an embodiment of the present application, wherein the matrix is the first target matrix with M=32 rows and J=48 columns, is a second target matrix with J=48 rows and N=32 columns. When the maximum matrix multiplication operation order in the current parallel computing hardware is 16×16, the first target matrix can be The first initial matrix A _single is divided into 2×3 with 16 rows and 16 columns; similarly, the second target matrix The second initial matrix B _single is divided into 3×2 matrices with 16 rows and 16 columns.

Step S102: Perform half-precision processing based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix.

In some embodiments, due to the limitation of some parallel computing hardware, the matrix multiplication operation device only supports half-precision matrix multiplication operations. Therefore, it is necessary to perform half-precision processing on the first initial matrix A _single and the second initial matrix B _single based on the single-precision data type of the first initial matrix A _single and the second initial matrix B _single to obtain a first half-precision matrix A _half of the first initial matrix A _single and a second half-precision matrix B _half of the second initial matrix B _single , so that the subsequent matrix multiplication operation device can perform matrix multiplication operations B half according to the first half-precision matrix A _half and the second half-precision matrix B _half .

In some embodiments, the first initial matrix and the second initial matrix are converted into a first half-precision matrix and a second half-precision matrix. After the matrix, since the half-precision floating-point data cannot fully represent the single-precision floating-point data, that is, when the single-precision data (FP32) input is converted into half-precision data (FP16), the mantissa part cannot be fully represented, which will introduce data truncation errors. Therefore, at this time, directly multiplying the first half-precision matrix and the second half-precision matrix as the result of the first initial matrix and the second initial matrix will result in lower accuracy. In order to explain the above-mentioned data truncation error and explain the present application at the same time, the method provided by the present application is further introduced with reference to Figure 4.

Referring to Fig. 4, it is a schematic diagram of a floating point data structure provided by an embodiment of the present application. Wherein, the first initial matrix A _single is a single precision matrix, so for each element data therein, there are 32 bits of data, of which 1 bit is used for the sign bit, 8 bits are used for the exponent, and 23 bits are used for the mantissa; in addition, the first half-precision matrix A _half is a half-precision matrix, so for each element data therein, there are 16 bits of data, of which 1 bit is used for the sign bit, 5 bits are used for the exponent, and 10 bits are used for the mantissa. Therefore, the first half-precision matrix A _half cannot completely store all the mantissa data of the first initial matrix A _single , that is, the first half-precision matrix A _half can only store the first mantissa part l ₁ of the first initial matrix A _single , which inevitably leads to data truncation errors. Therefore, in the embodiment of the present application, the first difference matrix RA _-half will be introduced, the purpose of which is to store the remaining second mantissa part l ₂ of the first initial matrix A _single . It can be understood that since the first difference matrix is also a half-precision matrix, its mantissa part can also store 10 bits of data. In addition, considering the hidden mantissa bits in the IEEE-754 floating point number rules, the first difference matrix actually stores the last ten mantissas of the mantissa part that is not 0 in the second mantissa part l _2. Therefore, theoretically, for the worst case, that is, the first data in the second mantissa part l ₂ is not 0, the mantissa data that can be stored in the first half-precision matrix and the first difference matrix reaches 21 bits, which is very close to the 23-bit mantissa data part of the first initial matrix. Based on this, the data truncation error that exists after the first initial matrix and the second initial matrix are converted into the first half-precision matrix and the second half-precision matrix can be effectively solved.

Therefore, the first difference matrix is used to save the error after the first initial matrix is converted to the first half-precision matrix, and the second difference matrix is used to save the error after the second initial matrix is converted to the second half-precision matrix, so that an error compensation term is added to the multiplication operation of the first half-precision matrix and the second half-precision matrix to perform corresponding half-precision multiplication operations, thereby obtaining a single-precision multiplication result with higher accuracy on a hardware device that only supports half-precision multiplication operations.

Step S103: obtaining a first difference matrix based on the difference between the first initial matrix and the first half-precision matrix, and obtaining a second difference matrix based on the difference between the second initial matrix and the second half-precision matrix.

In some embodiments, in order to solve the data truncation error after the first initial matrix and the second initial matrix are converted into the first half-precision matrix and the second half-precision matrix, the matrix multiplication operation device obtains a first difference matrix based on the difference between the first initial matrix and the first half-precision matrix, and obtains a second difference matrix based on the difference between the second initial matrix and the second half-precision matrix. It can be understood that, due to the limitation of some parallel computing hardware, the matrix multiplication operation device only supports multiplication operations of half-precision matrices, so the first difference matrix and the second difference matrix are both half-precision matrices.

In some embodiments, in order to more effectively improve the accuracy of single-precision matrix multiplication operations when using a computing chip that supports half-precision calculations, it is necessary to generate the first difference matrix and the second difference matrix more accurately. The process of generating the first difference matrix and the second difference matrix provided in an embodiment of the present application is described below.

5 , the process of obtaining a first difference matrix based on the difference between a first initial matrix and a first half-precision matrix includes steps S501 and S502 .

Step S501: Perform single-precision processing on the first half-precision matrix to obtain a first intermediate matrix.

Step S502: performing half-precision processing on the difference between the first initial matrix and the first intermediate matrix to obtain a first difference matrix.

In some embodiments, in order to more accurately generate the first difference matrix R _A-half , it is necessary to accurately determine the difference between the first half-precision matrix A _half and the first initial matrix A _single , so it is necessary to perform single-precision processing on the first half-precision matrix A _half to obtain the first intermediate matrix to_single(A _half ). Then, according to the first initial matrix A _half and the first intermediate matrix The difference A _half -to_single _{(A half )} is processed with half precision to obtain a first difference matrix with higher precision:
R _A-half =to_half(A _half -to_single(A _half )) (1)

In some embodiments, similar to the first difference matrix, in order to more accurately generate the second difference matrix RB _-half , it is necessary to accurately determine the difference between the second half-precision matrix _Bhalf and the second initial matrix _Bsingle , so the second half-precision matrix _Bhalf needs to be processed with single precision to obtain the second intermediate matrix to_single( _Bhalf ). Then, half-precision processing is performed based on the difference _Bhalf -to_single( _Bhalf ) between the second initial matrix _Bhalf and the second intermediate matrix to_single( _Bhalf ) to obtain a second difference matrix with higher precision:
R _B-half =to_half(B _half -to_single(B _half )) (2)

Step S104: Accumulate the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and use the first single-precision target matrix as the result of matrix multiplication of the first initial matrix and the second initial matrix.

In some embodiments, after obtaining the first difference matrix and the second difference matrix, in order to more effectively improve the accuracy of the single-precision matrix multiplication operation in a computing chip that supports half-precision calculation, the multiplication operation of the first initial matrix and the second initial matrix can be further converted into:

It can be understood that formula (3) includes the product A _half B _{half of} the first half-precision matrix and the second half-precision matrix, and also includes the compensation term A _half RB _-half +B _half RA _-half + _RA-half RB _-half .

In some embodiments, considering that the first difference matrix RA _-half is used to store the difference between the first initial matrix A _{single and} the first half-precision matrix A _half , the value of the first difference matrix RA- _half will be relatively small compared with the first half-precision matrix _{A half} ; similarly, considering that the second difference matrix RB _-half is used to store the difference between the second initial matrix B _{single and} the second half-precision matrix B _half , the value of the second difference matrix RB _{- half} will be relatively small compared with the second half-precision matrix B half. Based on this, for formula (3), the product RA _-half RB _-half of the first difference matrix and the second difference matrix is very small compared with other terms. Therefore, in order to improve the processing efficiency of the processing method of matrix multiplication operation in parallel computing hardware in the embodiment of the present application, the product RA _-half RB _-half of the first difference matrix and the second difference matrix in formula (3) can be ignored as a redundant term. Based on this, the first single-precision target matrix, which is the result of the matrix multiplication operation of the first initial matrix and the second initial matrix, can be obtained by accumulating the product of the first half-precision matrix A _half and the second _{half-precision matrix B half ,} the product of the first half-precision matrix _{A half and the second difference matrix RB-half} , and the product of the second half-precision matrix A _half and the first difference matrix RA _-half :

In some embodiments, for some application scenarios that place more emphasis on single-precision matrix multiplication operations than processing efficiency, in order to more effectively improve the accuracy of single-precision matrix multiplication operations in a computing chip that supports half-precision calculations, the product RA _{-half and} RB _-half (i.e., redundant terms) of the first difference matrix and the second difference matrix in formula (3) still need to be considered.

Therefore, referring to FIG6 , the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix are accumulated to obtain a first single-precision target matrix, including steps S601 to S603 .

Step S601: Accumulate the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first addition term.

Step S602: Obtain a second addition term according to the product of the first difference matrix and the second difference matrix.

Step S603: Accumulate the first addition term and the second addition term to obtain a first single-precision target matrix.

In some embodiments, in order to more effectively improve the accuracy of single-precision matrix multiplication operations in a computing chip that supports half-precision calculations, based on formula (3), the product A _half B _half of the first half-precision matrix and the second half-precision matrix, the product A _half R _B-half of the first half-precision matrix and the second difference matrix, and the product B _half R _A-half of the second half-precision matrix and the first difference matrix are accumulated to obtain a first addition term represented as follows:
C _{single_1} ＝A _half B _half +A _half R _B-half +B _half R _A-half (5)
According to the product of the first difference matrix and the second difference matrix RA _-half R _B-half , the second addition term is expressed as follows:
C _{single_2} ＝R _A-half R _B-half (6)

The first addition term C _{single_1} and the second addition term C _{single_2} are accumulated to obtain the first single-precision target matrix as follows:

In some embodiments, in the IEEE-754 standard, if the magnitude of two floating-point numbers differs too much, when the two floating-point numbers are added, the smaller floating-point number may be ignored, which is a phenomenon of "floating-point underflow". Taking F32 with a mantissa of 23 bits as an example, =8388608, the length is 7, which means that FP32 can represent up to 7 significant digits (the number of digits after the decimal point), the 7th bit may not represent all, but the 6th bit must be valid. Assume that a＝112345.1, b＝0.00001, in mathematics, there will be a+b＝112345.10001, but the result of programming calculation is a+b＝112345.101562, and the smaller floating point number is partially ignored; this result occurs because 112345.10001 must be converted into 1.1234510001 in the IEEE-754 single-precision floating point number. Since the maximum number of significant digits after the decimal point is 7, 1.1234510001 is inaccurate from the 8th decimal point. In addition, if two numbers are close, when the two numbers are subtracted, the result of the subtraction of the two numbers will be very small. If the result is too small to be represented, the smaller result will be rounded to zero.

With respect to the analysis of formula (7), on the one hand, if the first initial matrix A _single and the first half-precision matrix A _half are very close, the first difference matrix RA _-half will be very small, which will indirectly cause B _half RA _-half in formula (7) to be very small; similarly, if the second initial matrix B _single and the second half-precision matrix B _half are very close, the second difference matrix RB _-half will be very small, which will indirectly cause A _half RB _-half in formula (7) to be very small; on the other hand, the amount of data of the remaining second mantissa part _l2 of the first initial matrix A _single recorded using the first difference matrix RA _-half is more than 10 orders of magnitude smaller than the amount of data of the second half-precision matrix B _half , which will further cause the floating point underflow problem to occur when performing the B _half RA _-half operation; similarly, the amount of data of the remaining second mantissa part l _B-2 of the second initial matrix B _single recorded using the second difference matrix RB _- half is more than 10 orders of magnitude smaller than the amount of data of the first half-precision matrix A _half , which will further cause the A _half RB-2 operation to be performed. Floating point underflow may also occur during _B-half operations.

Therefore, in order to solve the floating-point underflow problem and to more effectively improve the accuracy of single-precision matrix multiplication operations in a computing chip that supports half-precision calculations, the first difference matrix RA _-half and the second difference matrix RB _-half need to be further processed accordingly.

Therefore, referring to Figure 7, the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix are accumulated to obtain the first single-precision target matrix, including steps S701 to S705.

Step S701: Obtain a preset multiplication value determined according to the number of digits in the mantissa.

Step S702: multiplying the first difference matrix by a preset multiplier to obtain a first magnified matrix, and multiplying the second difference matrix by a preset multiplier to obtain a second magnified matrix.

Step S703: Accumulate the product of the first half-precision matrix and the second magnification matrix, and the product of the second half-precision matrix and the first magnification matrix to obtain an intermediate magnification matrix.

Step S704: Divide the intermediate magnification matrix by a preset multiplication value to obtain an intermediate reduction matrix.

Step S705: Accumulate the product of the first half-precision matrix and the second half-precision matrix and the intermediate reduction matrix to obtain a first single-precision target matrix.

In some embodiments, since the first half-precision matrix A _half is used to store the first mantissa part l ₁ of the first initial matrix As _single , and the first difference matrix RA _-half is used to store the remaining second mantissa part l ₂ of the first initial matrix A _single , and due to the hidden mantissa bits in the IEEE-754 floating point number rule. Therefore, the data values of the first initial matrix A _single and the data values of the first difference matrix RA _-half actually differ by at least 11 orders of magnitude. For the above reasons, the preset multiplication value is determined to be 2 ¹¹ according to the number of mantissa bits of the half-precision data structure. Then, the preset multiplication value 2 ¹¹ is multiplied by the first difference matrix RA _-half to obtain the first enlarged matrix:
2 ¹¹ R _A-half =to_half(A _half -to_single(A _half )×2 ¹¹ ) (8)

At this time, the data values of the first magnified matrix 2 ¹¹ RA _-half and the data values of the second half-precision matrix B _half are of similar order of magnitude, which can effectively prevent the floating point underflow problem of B _half (2 ¹¹ RA _-half ). Similarly, the second difference matrix is multiplied by the preset multiplication value to obtain the second magnified matrix:
2 ¹¹ R _B-half =to_half(B _half -to_single(B _half )×2 ¹¹ ) (9)

At this time, the data values of the second magnified matrix 2 ¹¹ R _B-half and the data values of the first half-precision matrix A _half are of similar order of magnitude, which can effectively prevent the floating point underflow problem of A _half (2 ¹¹ R _B-half ). Next, based on the intermediate term A _half R _B-half +B _half R _A-half of formula (7), the product of the first half-precision matrix and the second magnified matrix, and the product of the second half-precision matrix and the first magnified matrix are further accumulated to obtain the intermediate magnified matrix:

Compared with formula (4), the intermediate magnification matrix is magnified by a preset multiplication value of 2 ¹¹ at the same time. Therefore, after solving the floating point underflow problem, the intermediate magnification matrix needs to be divided by the preset multiplication value 2 ¹¹ to obtain the intermediate reduction matrix:
(A _half (2 ¹¹ R _B-half )+B _half (2 ¹¹ R _A-half ))/2 ¹¹ (11)
At this time, based on formula (4), the product of the first half-precision matrix and the second half-precision matrix and the intermediate reduction matrix are accumulated.

The first single-precision target matrix is obtained as:
C _single ≈A _half B _half +(A _half (2 ¹¹ R _B-half )+B _half (2 ¹¹ R _A-half ))/2 ¹¹ (12)

At this time, it can effectively avoid the floating-point underflow in the A _{half RB} -half operation due to the large gap between the data values of the first difference matrix RB _-half and the second half-precision matrix B _half ; at the same time, it can also effectively avoid the floating-point underflow in the B _half RB-half operation due to the large gap between the data values of the second difference matrix _RB-half and the first half-precision matrix _{A half} , thereby more effectively improving the accuracy of single-precision matrix multiplication operations in computing chips that support half-precision calculations.

In some embodiments, when the data scale of the first target matrix and the second target matrix is large, the first target matrix needs to be divided into multiple first initial matrices, and the second target matrix needs to be divided into multiple second initial matrices. At this time, the matrix multiplication process of the first target matrix and the second target matrix is shown in FIG3. First, the second initial matrix is selected in the second target matrix according to the block position of the first initial matrix in the first target matrix, and multiple matrix groups are generated as follows:

Where k represents the number of the first initial matrix and the second initial matrix. Then, after performing matrix operations on the first initial matrix and the second initial matrix, multiple first single-precision target matrices C _single can be obtained; finally, all the first single-precision target matrices C _single are accumulated to obtain a second single-precision target matrix after multiplying the first target matrix and the second target matrix. At this time, the second single-precision target matrix can be obtained as follows:

At this time, combining formula (12) and formula (14), when the first target matrix is divided into multiple first initial matrices, and the second target matrix is divided into multiple second initial matrices, as shown in Figure 8, it is a flow chart of a matrix block multiplication operation provided by an embodiment of the present application, wherein the first initial matrix and the second initial matrix in each matrix sequence are operated in sequence according to formula (12) to obtain a first single-precision matrix C _single , and then the first single-precision matrix C _single is accumulated to obtain the second single-precision matrix C _single .

It is understandable that in order to further improve the accuracy of the matrix multiplication operation, the process shown in FIG8 should be added with redundant items RA _-half RB _-half , as shown in FIG9. FIG9 is another schematic diagram of a process of a matrix block multiplication operation provided by an embodiment of the present application, wherein, compared with FIG8, it is necessary to perform a plurality of redundant items RA _-half RB _-half multiplication operations.

In some embodiments, when the data scale of the first target matrix and the second target matrix is small, that is, when the maximum matrix multiplication operation order in the parallel computing hardware is larger than the data scale of the first target matrix and the second target matrix, it is further explained that the multiplication operation of the first target matrix and the second target matrix can be processed immediately in the parallel computing hardware. At this time, the first target matrix is directly used as the first initial matrix, and the second target matrix is also used as the second initial matrix. It can be understood that at this time, the first single-precision target matrix C _single is equal to the second single-precision target matrix C _single .

In some embodiments, when the operation flow shown in FIG8 is used, there is still a floating point underflow caused by the large order of magnitude difference between different first single-precision matrices C _single . In addition, considering that the data value of A _half B _half in formula (13) is significantly different from the data value of (A _half (2 ¹¹ RB _-half )+B _half (2 ¹¹ RA _-half ))/2 ¹¹ , different sizes of storage cache spaces need to be divided for these two items in the parallel computing hardware. If the operation flow shown in FIG8 is still used, when performing the matrix multiplication operation of the first target matrix and the second target matrix, the parallel computing hardware needs to divide different storage cache spaces multiple times to store them in sequence. Data and This will limit the efficiency of performing matrix multiplication operations in parallel computing hardware.

Therefore, in order to improve the accuracy of single-precision matrix multiplication operations, and also to improve the work efficiency of performing matrix multiplication operations in parallel computing hardware, it is necessary to further optimize the operation process shown in Figure 8 as follows. Referring to Figure 10, the processing method of matrix multiplication operations in parallel computing hardware provided in the embodiment of the present application also includes the following steps S1001 to S1003.

Step S1001: selecting a second initial matrix in a second target matrix according to the block positions of the first initial matrix in the first target matrix, and generating a plurality of matrix sequences.

Step S1002: Accumulate the product of the first half-precision matrix and the second half-precision matrix in each matrix group to obtain a first cumulative matrix of the matrix sequence, and accumulate the sum of the product of the first half-precision matrix and the second difference matrix and the product of the second half-precision matrix and the second difference matrix in each matrix group to obtain a second cumulative matrix of the matrix sequence.

Step S1003: Obtain a second single-precision target matrix according to the first accumulation matrix and the second accumulation matrix, and use the second single-precision target matrix as the result of matrix multiplication of the first target matrix and the second target matrix.

In some embodiments, as in the above operation, when the first target matrix is divided into a plurality of first initial matrices, and the second target matrix is divided into a plurality of second initial matrices, then referring to formula (1) and FIG. 3 , the second initial matrix is selected in the second target matrix according to the block position of the first initial matrix in the first target matrix, and a plurality of matrix sequences are generated, wherein each matrix sequence includes a plurality of matrix groups, each matrix group includes a first initial matrix and a second initial matrix corresponding to the first initial matrix, wherein the plurality of matrix groups are as shown in formula (14).

Next, referring to FIG11 , there is a schematic diagram of an improved process flow of a matrix block multiplication operation provided by an embodiment of the present application. In which, the product of the first half-precision matrix and the second half-precision matrix in each matrix group is accumulated to obtain the first cumulative matrix of the matrix sequence, as shown below:

Next, the product of the first half-precision matrix and the second difference matrix in each matrix group and the product of the second half-precision matrix and the second difference matrix in each matrix group are accumulated to obtain the second cumulative matrix of the matrix sequence as shown below:

Then, the first accumulation matrix and the second accumulation matrix are accumulated to obtain the second single-precision target matrix C _single . This effectively avoids the floating-point underflow caused by the large order of magnitude difference between different first single-precision matrices C _single , thereby improving the accuracy of single-precision matrix multiplication operations. At the same time, the parallel computing hardware only needs to divide the storage cache space into two different sizes when performing the matrix multiplication operation of the first target matrix and the second target matrix. In addition, the parallel computing capability of the parallel computing hardware can be better utilized, that is, multiple items can be calculated simultaneously. Operations and multiple operations, thereby effectively improving the work efficiency of single-precision matrix multiplication operations.

It is understandable that in order to further improve the accuracy of the matrix multiplication operation, the process shown in FIG11 should be added with redundant items RA _-half RB _-half , as shown in FIG12. FIG12 is a schematic diagram of another improved process of a matrix block multiplication operation provided by an embodiment of the present application, wherein, compared with FIG11, it is necessary to perform a plurality of redundant items RA _-half RB _-half multiplication operations.

In some embodiments, in order to improve the reliability of the processing method of matrix multiplication operation in parallel computing hardware, it is also necessary to add a detection and comparison link. Therefore, referring to Figure 11, the processing method of matrix multiplication operation in parallel computing hardware also includes the following steps S1301 to S1304.

Step S1301: Perform double-precision processing based on single-precision data to obtain a first double-precision matrix of the first target matrix, a second double-precision matrix of the second target matrix, and a check matrix of the second single-precision target matrix.

Step S1302: performing a multiplication operation on the first double precision matrix and the second double precision matrix to obtain an evaluation matrix.

Step S1303: Obtaining a test result based on the test matrix and the evaluation matrix.

Step S1304: Compare the inspection result with a preset inspection threshold, and output a second single-precision target matrix based on the comparison result.

In some embodiments, in order to improve the reliability of the processing method of matrix multiplication operations in parallel computing hardware, and also to verify the accuracy of the second single-precision target matrix C _single obtained by using the processing method of matrix multiplication operations in parallel computing hardware provided by the present application. Utilizing the high precision of the double-precision data structure, after obtaining the second single-precision target matrix C _single , the matrix multiplication operation device performs double-precision processing on the second single-precision target matrix C _single based on the single-precision data structure of the second single-precision target matrix C _single , and obtains a test matrix to_double(C _single ) of the second single-precision target matrix C _single ; at the same time; and based on the _{single-precision data structure of the first target matrix A single ,} the first target matrix A _single is double-precision processed to obtain a first double-precision matrix to_double(A _single ) of the first target matrix A _single , and based on the single-precision data structure of the second target matrix B _single , the second target matrix B _single is double-precision processed to obtain a second double-precision matrix to_double(B _single ) of the second target matrix B single, and then double-precision matrix multiplication is performed on the first double-precision matrix and the second double-precision matrix to obtain an evaluation matrix to_double(A _single )·to_double(B _single ). Next, a test is performed based on the test matrix and the evaluation matrix as follows:

Wherein, V is used to store the relative residual between the test matrix and the evaluation matrix, and is used to characterize the actual error between the processing method of the matrix multiplication operation in the parallel computing hardware provided by this embodiment and the direct matrix multiplication operation of the first target matrix and the second target matrix; ||·|| _F is the Euclidean norm of the matrix, which can be understood as the Euclidean norm of the matrix The number refers to the largest singular value of a matrix, also known as the 2-norm of the matrix, which is used to evaluate the condition number of the matrix, that is, the stability of the matrix and the reliability of the numerical solution.

When the relative residual V is less than the preset test threshold, it can be determined that the processing method for matrix multiplication operation in the parallel computing hardware provided in the embodiment of the present application is effective, so the second single-precision target matrix can be used as the result of the matrix multiplication operation of the first target matrix and the second target matrix and output. In the embodiment of the present application, no constraints are imposed on the setting of the preset test threshold, that is, it can be artificially preset, or it can be obtained by the matrix multiplication operation device according to the historical operation law.

It can be understood that the verification steps shown in steps S1301 to S1304 are for verifying the reliability of the processing method for matrix multiplication operations in the parallel computing hardware provided in the embodiment of the present application, and the processing method for matrix multiplication operations in the parallel computing hardware provided in the embodiment of the present application is mainly used for matrix multiplication operations of two single-precision matrices (i.e., the first target matrix and the second target matrix) in parallel computing hardware that only supports half-precision matrix multiplication operations. Therefore, in this case, the verification steps shown in steps S1301 to S1304 will be placed in other parallel computing hardware that can support double-precision matrix multiplication operations for execution, and will have no effect on the processing method for matrix multiplication operations in the parallel computing hardware provided in the embodiment of the present application.

In some instances, the processing method of matrix multiplication operation in parallel computing hardware provided in this application can be applied to the Ascend AI processor, which is a chip adapted to a specific field, and the core of Ascend AI processing is an artificial intelligence chip. As shown in Figure 14, it is a schematic diagram of the chip structure of the Ascend AI processor. The Ascend AI processor provides three basic computing units: matrix computing unit (CUBE), vector computing unit (Vector) and scalar computing unit (Scalar). The three computing units will form three independent pipelines for the corresponding calculations to complete the corresponding calculations. Among them, the L1 buffer is used to store the first target matrix and the second target matrix, and then perform a matrix conversion operation and save it in the cache conversion unit, wherein the conversion operation includes dividing the first target matrix into a plurality of first initial matrices, and converting the first initial matrix into a first half-precision matrix, and generating a first difference matrix and a first magnification matrix. Similarly, the second target matrix is divided into a plurality of second initial matrices, and the second initial matrix is converted into a second half-precision matrix, and a second difference matrix and a second magnification matrix are generated. Buffer L0A and buffer L0B are used to store matrices that are about to perform matrix multiplication operations. The matrix calculation unit is used to receive the matrices in the buffer L0A and the buffer L0B to perform a matrix multiplication operation. In conjunction with the execution flow chart shown in FIG12 , the matrix multiplication operation includes: as well as The accumulator is used to accumulate the data obtained by the matrix calculation unit, that is, the addition operation shown in Figure 12. The buffer L0C is used to store the data obtained by the accumulator calculation, and finally obtain the second single-precision target matrix C _single required by this solution.

In the artificial intelligence chip of the Ascend AI processor, the unified buffer is an important component inside it, which is used to store data shared between different computing units. The system control module is a module in the Ascend chip, which is responsible for controlling the overall operation and scheduling of the chip. It includes the coordination and management between various functional modules, as well as tasks such as processing external input and output. The bus interface module is an interface module used to connect the Ascend chip with other external devices. It provides an interface for data transmission and communication with the host system or other external devices, and realizes interaction with external systems. The instruction cache is a high-speed cache used to store instructions in the Ascend chip. It is used to increase the reading speed of instructions, reduce instruction access latency, and improve the execution efficiency of instructions. The scalar instruction processing queue is a module in the Ascend chip, which is used to process scalar instructions. A scalar instruction is an instruction that operates on a single data, such as addition, multiplication, etc. The scalar instruction processing queue is responsible for receiving and decoding scalar instructions and distributing them to the corresponding functional units for execution. The instruction distribution module is a module in the Ascend chip, which is used to distribute the decoded instructions to the corresponding functional units for execution. It is responsible for distributing instructions to appropriate functional units according to their type and operands, realizing parallel execution of instructions and efficient use of computing resources. In summary, the unified buffer in the Ascend chip is used to store data shared between different computing units, the system control module is responsible for overall scheduling and management, the bus interface module realizes communication with external devices, the instruction cache improves the instruction reading speed, the scalar instruction processing queue processes scalar instructions, and the instruction distribution module distributes the decoded instructions to the corresponding functional units for execution. These modules together constitute part of the functions in the Ascend chip. The CUBE queue is a hardware module in the Ascend chip, which is used to manage and schedule the execution of tasks. The CUBE queue can execute multiple tasks in parallel, and switch and schedule tasks according to certain scheduling strategies to improve computing efficiency. The Vector queue is another hardware module in the Ascend chip, which is used to support vector computing. Vector queue The column can efficiently perform vector operations and improve the efficiency and performance of vector computing by processing multiple vector data in parallel. The storage conversion queue is a hardware module in the Ascend chip, which is used to handle data storage and conversion. The storage conversion queue is responsible for managing data transmission and conversion between different storage media, such as data reading and writing between memory and external storage devices. The time synchronization module is a hardware module in the Ascend chip, which is used to ensure time synchronization between multiple Ascend chips. The time synchronization module uses a precise clock synchronization mechanism to ensure that multiple Ascend chips have a consistent time base when performing tasks to support distributed computing and collaborative work. The vector computing unit is a hardware module in the Ascend chip, which is used to perform vector computing. The vector computing unit can efficiently perform operations on large-scale vector data, provide parallel computing capabilities, and is used to accelerate vector computing-intensive tasks. The scalar computing unit is a hardware module in the Ascend chip, which is used to perform scalar computing. The scalar computing unit is responsible for processing the computing tasks of a single data element, including addition, subtraction, multiplication, division, logical operations, etc., to support scalar computing-intensive tasks. The configuration port is an interface in the Ascend chip, which is used to configure and manage various parameters and settings of the chip. The configuration port provides a communication channel with the internal control logic of the chip, which is used to read and write the configuration registers of the chip to achieve flexible configuration of the chip functions and performance. The bus is a communication channel in the Ascend chip, which is used to connect various modules inside the chip and external devices. The bus is responsible for transmitting data and control signals to achieve communication and collaboration between various modules inside the chip. The L2 cache area is a high-speed cache storage area in the Ascend chip, located between the chip core and the memory. The L2 cache area is used to store frequently accessed data and instructions, providing fast data reading and writing and access speed to speed up computing and data processing. DDR (Double Data Rate) is a type of memory in the Ascend chip and one of the common memory types in computer systems. DDR memory uses double data transfer rate technology, which can transmit twice the amount of data in one clock cycle, providing higher memory bandwidth and faster data access speed, and is used to store and read and write large-scale data.

It can be understood that the execution steps of the main improvements of the present application are mainly in the L1 buffer, cache conversion unit, buffer L0A, buffer L0B, buffer L0C, matrix calculation unit and accumulator in the artificial intelligence chip of the Ascend AI processor.

In some embodiments, the processing method of matrix multiplication in parallel computing hardware provided by the present application can be applied to ordinary parallel computing hardware for block execution processing. Parallel computing hardware usually includes multiple artificial intelligence chips, so each block multiplication operation can be processed in parallel using one artificial intelligence chip. For example, in the execution process shown in FIG12, the first artificial intelligence chip is used to perform the block matrix multiplication when i=1, and each block matrix multiplication can be processed by the corresponding artificial intelligence chip. as well as Execute in parallel, and then add up the obtained data at the end, so as to improve the work efficiency of performing matrix multiplication operations. Take the Ascend AI processor as an example. The Ascend AI processor supports 32 artificial intelligence chips, and theoretically can process 32 block matrix multiplication operations at the same time. Four artificial intelligence chips are selected in a certain Ascend AI processor for example, and the relevant description is further made with reference to Figure 15. Figure 15 is a schematic diagram of the execution of block parallel operations provided by an embodiment of the present application. The first target matrix and the second target matrix are divided into 4 blocks, the data length calculated by each artificial intelligence chip and the index value of the block are calculated, and then the calculations are performed in parallel by 4 artificial intelligence chips, and finally the calculation results of each artificial intelligence chip are summarized, and the effect of accelerating the matrix calculation time is achieved through the parallel calculation of multiple artificial intelligence chips. Compared with the case of using one artificial intelligence chip, the parallel calculation by 4 artificial intelligence chips can theoretically increase the work efficiency by four times.

In some embodiments, it can be seen from Figure 14 that when the core of the artificial intelligence chip of the Ascend AI processor performs matrix calculations, the input data is required to be placed in the buffer L0, and the output ultimately uses the cache unit UB. At the same time, the memory requested by the host side must be passed through the Global Memory (GM) to be passed to the Ascend CUBE core for use. Therefore, the impact of data transfer on computing performance must be fully considered.

When the amount of data after block division is small, data handling is the main factor affecting performance. In order to fully utilize the high-speed internal cache UB of only 256KB on the Ascend AI processor, the calculation within a block must apply for different memory sizes on the UB multiple times, which will limit the size of the data block on the UB and cause the data on the UB to be exchanged to the temporary GM for storage, which will reduce the execution efficiency. To solve this problem, the same calculation can be modularized. That is, α ^[k] and β ^[k] are calculated separately, so that UBs of the same size can be applied for uniformly, reducing the number of UB applications. At the same time, it will also reduce the number of data transfers between GM and UB, and improve execution efficiency.

In some embodiments, when the amount of input data involved in matrix multiplication is particularly large, the time spent on calculation exceeds the time spent on data transfer. Blocking can only solve the size limit of UB, but cannot solve the time-consuming problem. At this time, the multi-core processing capabilities of Ascend AI must be fully utilized. For example: Assume that for the test matrix (M, N, K), the calculation of K is performed on multiple cores for parallel calculation. This involves accumulating the calculation results of all cores for the same block after the multi-core calculation is completed. Therefore, the intermediate results of the two calculations must be stored separately in advance.

The Ascend AI processor achieves acceleration through large-scale parallel computing capabilities. It uses the Bisheng C++ language tool to express the mapping to the device's computing core through the concept of a parallel computing workgroup. Each parallel computing workgroup is mapped to a specific core. Each parallel computing workgroup has the same instruction code but a different identification ID, which is similar to a SPMD (Single Program Multiple Data) technology.

SPMD is a parallel computing model, which means that multiple processors or computing units execute the same program at the same time, but corresponding to different data. In the SPMD model, different processors or computing units have their own data sets and independently execute the same instruction sequence to process these data. The SPMD model is a model based on task parallelism, which is suitable for many parallel computing applications, such as scientific computing, image processing, and data analysis. In the SPMD model, programmers need to divide the computing problem into multiple independent tasks and assign different data sets to each task. Then, each processor or computing unit will execute the same program in parallel, but corresponding to different data sets.

As shown in reference figure 16, it is a precision comparison simulation diagram of the processing method of matrix multiplication operation in parallel computing hardware provided by the embodiment of the present application. In order to further verify the reliability of the results obtained by adopting the processing method of matrix multiplication operation in parallel computing hardware provided by the embodiment of the present application, the present embodiment compares the NPU verification result (this patent solution), that is, the matrix multiplication operation processing solution provided by the present application with the other two solutions, namely: 1. NPU verification result (no optimization solution): that is, as in the related art, directly divide the first target matrix and the second target matrix into multiple first initial matrices and multiple second initial matrices, and then convert the divided multiple first initial matrices and multiple second initial matrices into corresponding multiple half-precision matrices and then perform half-precision matrix multiplication operation to obtain the result; 2. Sgemm verification result solution of OpenBLAS library: that is, as in the related art, directly use the sgemm of OpenBLAS library to perform single-precision matrix multiplication operation on the first target matrix and the second target matrix. It can be understood that the OpenBLAS library is an open source basic linear algebra subroutine (BLAS) library for high-performance numerical calculations. BLAS is a set of standard interfaces and functions for performing common linear algebra operations such as matrix multiplication, vector addition, etc. In addition, in the OpenBLAS library, sgemm is a function for performing matrix multiplication.

The horizontal axis in Figure 16 refers to the size of the input matrix k (i.e., the number of columns K of the first target matrix and the number of rows K of the second target matrix), and the vertical axis refers to the accuracy test difference. It can be clearly seen that compared with the NPU verification result (without optimization solution), the relative residual V of the second single-precision target matrix C _single obtained by the NPU verification result (this patent solution) is very low, and the accuracy of the second single-precision target matrix C _single obtained by adopting the NPU verification result (this patent solution) is very close to the sgemm verification result solution of the OpenBLAS library, thereby verifying the reliability of the results obtained by adopting the processing method of matrix multiplication operations in parallel computing hardware provided by the embodiment of the present application.

Referring to Figures 17 and 18, there are a simulation diagram of the work efficiency comparison of the processing method for matrix multiplication operations in parallel computing hardware provided in the embodiment of the present application and a simulation data table diagram of the work efficiency of the processing method for matrix multiplication operations in parallel computing hardware. In order to further verify the reliability of the work efficiency improved by the processing method for matrix multiplication operations in parallel computing hardware provided by the embodiment of the present application, this embodiment simulates and compares the running time of the optimized calculation execution process (i.e., the execution process shown in Figure 11, i.e., the NPU verification result (this patent solution) in Figure 17) used when executing this solution with the running time of the sgemm verification result solution of the OpenBLAS library. The horizontal axis in Figure 17 refers to the size of the input matrix k (i.e., the number of columns K of the first target matrix and the number of rows K of the second target matrix), and the vertical axis refers to the calculation delay. It can be clearly seen that with the increase of the number of columns K of the first target matrix and the number of rows K of the second target matrix, the running time of the sgemm verification result solution of the OpenBLAS library will increase significantly, while the running time of the NPU verification result (patent solution) (i.e., the execution flow shown in FIG. 11 ) increases, but its The increase is very small. In addition, combined with the acceleration ratio data of Figure 18, it can be clearly obtained that when the size of the input matrix k (that is, the number of columns K of the first target matrix and the number of rows K of the second target matrix) reaches 2 ¹¹ or more, the running time of the NPU verification result (this patent solution) is much less than the running time of the sgemm verification result solution of the OpenBLAS library. It can be understood that the acceleration data is the ratio of the running time of the sgemm verification result solution of the OpenBLAS library to the running time of the NPU verification result (this patent solution). Thereby verifying the reliability of the work efficiency improved by the optimized calculation execution process (that is, the execution process shown in Figure 11) adopted in the processing method of matrix multiplication operation in the parallel computing hardware provided by the embodiment of the present application.

FIG19 is another precision comparison simulation diagram of the processing method for matrix multiplication operation in parallel computing hardware provided by an embodiment of the present application. In order to further verify the reliability of the work efficiency improved by adopting the processing method for matrix multiplication operation in parallel computing hardware provided by an embodiment of the present application. This embodiment compares the two schemes shown in FIG11 and FIG12, namely, the redundant items Schemes and non-redundant items The schemes are compared with each other in terms of the accuracy (i.e., relative residuals). The horizontal axis in Figure 19 is the size of the input matrix k (i.e., the number of columns K of the first target matrix and the number of rows K of the second target matrix), and the vertical axis is the relative residual. It can be clearly seen that when the maximum difference in the relative residuals of the two schemes is around 10 ^-8 , the accuracy of the results obtained by the two schemes is basically the same. Therefore, when the execution flow scheme shown in Figure 11 is adopted, the amount of calculation can be effectively reduced by one quarter, thereby reducing the cache usage by one quarter, further improving the work efficiency of matrix multiplication operations.

An embodiment of the present application provides a method for processing matrix multiplication operations in parallel computing hardware, which can improve the accuracy of single-precision matrix multiplication operations when using a computing chip that supports half-precision calculations. A first target matrix and a second target matrix, both of which are single-precision matrices, are divided into at least one first initial matrix and at least one second initial matrix, and a plurality of corresponding matrix groups are obtained; then half-precision processing is performed based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix in each matrix group; then a first difference matrix is obtained based on the difference between the first initial matrix and the first half-precision matrix in each matrix group, and a second difference matrix is obtained based on the difference between the second initial matrix and the second half-precision matrix; then the first difference matrix and the second difference matrix are multiplied by a preset multiplication value to obtain a first magnification matrix and a second magnification matrix; then the product of the first half-precision matrix and the second magnification matrix and the product of the second half-precision matrix and the first magnification matrix are accumulated to obtain an intermediate magnification matrix of each matrix group, and the intermediate magnification matrices of all matrix groups are accumulated to obtain a first accumulation matrix, and the products of the first half-precision matrices and the second half-precision matrices of all matrix groups are accumulated to obtain a second accumulation matrix, and the first accumulation matrix and the second accumulation matrix are added to obtain a second single-precision target matrix, and the second single-precision target matrix is used as the result of matrix multiplication operation between the first target matrix and the second target matrix.

The embodiment of the present application is directed to the process of performing single-precision matrix multiplication using a computing chip that supports half-precision calculation. First, based on the hardware parameters of the parallel computing hardware, the first target matrix and the second target matrix are divided to obtain multiple first initial matrices and second initial matrices, and the first initial matrix and the second initial matrix are processed with half precision to obtain the first half-precision matrix and the second half-precision matrix, so as to facilitate the subsequent execution calculation of the parallel computing hardware. Then, the error after the first initial matrix is converted to the first half-precision matrix is saved by using the first difference matrix, and the error after the second initial matrix is converted to the second half-precision matrix is saved by using the second difference matrix, so as to add an error compensation item in the multiplication operation of the first half-precision matrix and the second half-precision matrix to perform the corresponding half-precision multiplication operation, thereby effectively improving the accuracy of the half-precision multiplication operation of the single-precision matrix. Then, the corresponding operation is performed after multiplying the first difference matrix and the second difference matrix by the preset multiplication value, so as to solve the floating-point underflow problem and improve the accuracy of the matrix multiplication operation. Finally, the multiplication operations of each matrix group are separated to adjust the operation order, so as to further solve the floating-point underflow problem and improve the working efficiency of the matrix multiplication operation device. This effectively obtains a single-precision multiplication result with higher accuracy on a hardware device that only supports half-precision multiplication operations, thereby improving the work efficiency of multiplication operations.

The embodiment of the present application further provides a matrix multiplication operation device, which can implement the processing method of the matrix multiplication operation in the above parallel computing hardware. Referring to FIG. 20 , the device 2000 includes:

An acquisition module 210 is used to acquire a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices;

A half-precision conversion module 2020 is used to perform half-precision processing based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix;

The difference processing module 2030 is used to obtain a first difference matrix based on the difference between the first initial matrix and the first half-precision matrix, and to obtain a second difference matrix based on the difference between the second initial matrix and the second half-precision matrix; wherein the first difference matrix and the second difference matrix are both half-precision matrices;

The calculation module 2040 is used to accumulate the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and use the first single-precision target matrix as the result of matrix multiplication operation between the first initial matrix and the second initial matrix.

The specific implementation of the matrix multiplication operation device of this embodiment is basically consistent with the specific implementation of the processing method of the matrix multiplication operation in the above-mentioned parallel computing hardware, and will not be repeated here.

The present application also provides an electronic device, including:

at least one memory;

at least one processor;

at least one program;

The program is stored in the memory, and the processor executes the at least one program to implement the processing method of matrix multiplication operation in the parallel computing hardware implemented in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

Please refer to FIG. 21 , which schematically shows the hardware structure of an electronic device according to another embodiment. The electronic device includes:

The processor 2101 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

The memory 2102 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 2102 can store operating systems and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 2102, and the processor 2101 calls and executes the processing method of matrix multiplication operation in the parallel computing hardware of the embodiment of this application;

Input/output interface 2103, used to implement information input and output;

Communication interface 2104, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

A bus 2105 that transmits information between the various components of the device (e.g., the processor 2101, the memory 2102, the input/output interface 2103, and the communication interface 2104);

The processor 2101 , the memory 2102 , the input/output interface 2103 and the communication interface 2104 are connected to each other in communication within the device via the bus 2105 .

An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, it implements a processing method for matrix multiplication operations in the above-mentioned parallel computing hardware.

The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules/units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And/or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and/or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character "/" generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store programs.

The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims (10)

A method for processing matrix multiplication operations in parallel computing hardware, characterized by comprising: Obtain a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices; Perform half-precision processing based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; Obtaining a first difference matrix based on a difference between the first initial matrix and the first half-precision matrix, and obtaining a second difference matrix based on a difference between the second initial matrix and the second half-precision matrix; wherein both the first difference matrix and the second difference matrix are half-precision matrices; Accumulate the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and use the first single-precision target matrix as a result of matrix multiplication of the first initial matrix and the second initial matrix. The method for processing matrix multiplication operations in parallel computing hardware according to claim 1, wherein obtaining a first difference matrix based on the difference between the first initial matrix and the first half-precision matrix comprises: Performing single-precision processing on the first half-precision matrix to obtain a first intermediate matrix; Perform half-precision processing on the difference between the first initial matrix and the first intermediate matrix to obtain the first difference matrix. The method for processing matrix multiplication operations in parallel computing hardware according to claim 1, characterized in that the accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix comprises: Accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first addition term; Obtaining a second addition term according to the product of the first difference matrix and the second difference matrix; The first addition item and the second addition item are accumulated to obtain the first single-precision target matrix. The method for processing matrix multiplication operations in parallel computing hardware according to claim 1, characterized in that the elements in the first half-precision matrix include mantissas; the accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, further comprising: Obtaining a preset multiplication value determined according to the number of digits of the mantissa; Multiplying the first difference matrix by a preset multiplication value to obtain a first magnification matrix, and multiplying the second difference matrix by the preset multiplication value to obtain a second magnification matrix; Accumulate the product of the first half-precision matrix and the second magnification matrix, and the product of the second half-precision matrix and the first magnification matrix to obtain an intermediate magnification matrix; Dividing the intermediate magnification matrix by the preset multiplication value to obtain an intermediate reduction matrix; The product of the first half-precision matrix and the second half-precision matrix and the intermediate reduction matrix are accumulated to obtain the first single-precision target matrix. The method for processing matrix multiplication operations in parallel computing hardware according to claim 1, wherein obtaining the first initial matrix and the second initial matrix comprises: Get a first target matrix and a second target matrix; Obtaining a maximum matrix multiplication operation order of the parallel computing hardware; Based on the maximum matrix multiplication operation order, the first target matrix is divided to obtain at least one first initial matrix, and the second target matrix is divided to obtain at least one second initial matrix. The method for processing matrix multiplication operations in parallel computing hardware according to claim 5, characterized in that when When there are a plurality of the first initial matrix and the second initial matrix, the method further comprises: Selecting the second initial matrix in the second target matrix according to the block position of the first initial matrix in the first target matrix, and generating a plurality of matrix sequences; each of the matrix sequences includes a plurality of the matrix groups, and each of the matrix groups includes the first initial matrix and the second initial matrix; Accumulating the product of the first half-precision matrix and the second half-precision matrix in each of the matrix groups to obtain a first accumulated matrix of the matrix sequence, and accumulating the sum of the product of the first half-precision matrix and the second difference matrix and the product of the second half-precision matrix and the second difference matrix in each of the matrix groups to obtain a second accumulated matrix of the matrix sequence; A second single-precision target matrix is obtained according to the first accumulation matrix and the second accumulation matrix, and the second single-precision target matrix is used as a result of matrix multiplication operation of the first target matrix and the second target matrix. The method for processing matrix multiplication operations in parallel computing hardware according to claim 6, characterized in that the method further comprises: Perform double-precision processing based on the single-precision data type to obtain a first double-precision matrix of the first target matrix, a second double-precision matrix of the second target matrix, and a check matrix of the second single-precision target matrix; Performing a multiplication operation on the first double precision matrix and the second double precision matrix to obtain an evaluation matrix; Obtaining a test result based on the test matrix and the evaluation matrix; The test result is compared with a preset test threshold, and the second single-precision target matrix is output based on the comparison result. A matrix multiplication operation device, characterized in that the device comprises: An acquisition module, used to acquire a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices; A half-precision conversion module, used for performing half-precision processing based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; A difference processing module, configured to obtain a first difference matrix based on a difference between the first initial matrix and the first half-precision matrix, and to obtain a second difference matrix based on a difference between the second initial matrix and the second half-precision matrix; wherein both the first difference matrix and the second difference matrix are half-precision matrices; A calculation module is used to accumulate the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and use the first single-precision target matrix as a result of matrix multiplication of the first initial matrix and the second initial matrix. An electronic device, characterized in that it includes a memory and a processor, the memory storing a computer program, and characterized in that when the processor executes the computer program, it implements the processing method of matrix multiplication operation in the parallel computing hardware described in any one of claims 1 to 7. A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the method for processing matrix multiplication operations in parallel computing hardware as described in any one of claims 1 to 7 is implemented.