WO2025124574A1 - Computing system, method executed by computing system, and storage medium - Google Patents

Abstract

Disclosed in the present application is a computing system. The computing system comprises a general processing unit and a tensor processing unit, wherein the general processing unit comprises an instruction scheduling module and at least one computing unit; the instruction scheduling module is used for identifying a computing instruction, and sending, when the computing instruction is identified as a tensor operation instruction, the tensor operation instruction to the tensor processing unit; and the tensor processing unit is used for performing in-memory computing and/or non-in-memory computing on the basis of the tensor operation instruction. Further provided in the present application is a computing method executed by the computing system. The general processing unit and the tensor processing unit are combined together, such that the general processing unit is used to process general computing in a neural network, and the tensor processing unit is used to perform in-memory computing or non-in-memory computing, thereby ensuring generality and also improving computing power, and supporting complex and variable computing operators in the neural network.

Description

Computing system, method executed by the computing system, and storage medium

This application claims priority to the Chinese patent application filed with the China Patent Office on December 13, 2023, with application number 202311713616.9 and application name “Computing system, method executed by a computing system, and storage medium”, the entire contents of which are incorporated into this application by reference.

Technical Field

The present invention relates to the field of computer technology, and more specifically, to a computing system, a method executed by the computing system, and a storage medium.

Background Art

In the existing technology, artificial intelligence chip solutions for detection tasks and computing tasks are mainly divided into two categories. One is general-purpose processors, such as CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), GPU (Graphics Processing Unit), etc.; the other is acceleration processors specifically for artificial neural networks, such as Google's TPU (Tensor Processing Units). These chips all belong to the von Neumann architecture, and their computing and storage are separated. The two work together to complete data access and computing. However, since the design of the processor is mainly to improve the computing speed, and the storage is more focused on capacity improvement and cost optimization, there is a performance mismatch between "storage" and "computing", which leads to problems such as low memory access bandwidth, extended latency, and high power consumption, which are commonly referred to as the "storage wall" and "power consumption wall". The denser the memory access, the more serious the "wall" problem, and the more difficult it is to improve computing power.

As a new computing architecture, storage-computing integration fully integrates storage and computing, effectively overcomes the bottleneck of the von Neumann architecture, and can greatly improve computing power. However, in-memory computing generally adopts the AISC (application-specific integrated circuit) mode, which supports very limited operator types, and has poor programming flexibility and versatility, and cannot flexibly match the current development speed of neural network models. CPUs, GPUs, etc. have advantages in data-intensive general computing, but general computing has difficulty in dealing with complex and changeable computing operators.

Summary of the invention

In view of the above problems, the purpose of the present invention is to provide a computing system, a method executed by the computing system and a storage medium, which combine a general-purpose processor and a tensor processor together, use the general-purpose processor to process general-purpose calculations in neural networks and use the tensor processor to perform in-memory calculations and non-in-memory calculations, thereby reducing data movement and improving computing efficiency while ensuring versatility.

According to a first aspect of the present invention, there is provided a computing system, comprising a general-purpose processor and a tensor processor; wherein the general-purpose processor comprises an instruction scheduling module and at least one computing unit, the instruction scheduling module being used to identify computing instructions, and when the computing instructions are identified as tensor operation instructions, sending the tensor operation instructions to the tensor processor; the tensor processor being used to perform in-memory computing and/or non-in-memory computing according to the tensor operation instructions.

Preferably, the instruction scheduling module is further used to send the general operation instruction to the computing unit when identifying that the computing instruction is a general operation instruction; the computing unit is used to perform general calculation according to the general operation instruction.

Preferably, the computing unit comprises at least one computing core, and the computing core is used to perform general computing according to general operation instructions.

Preferably, the computing instruction includes an operation code and an operation field, wherein the operation code includes the instruction type and operation code of the computing instruction, and the operation field includes the source address and target address of the data to be operated.

Preferably, the instruction types include tensor operations, vector operations, scalar operations and transcendental function operations.

Preferably, when the instruction type in the computing instruction is a tensor operation, the instruction scheduling module identifies the computing instruction as a tensor operation instruction; when the instruction type in the computing instruction is one of a vector operation, a scalar operation and a transcendental function operation, the instruction scheduling module identifies the computing instruction as a general operation instruction.

Preferably, the tensor operation instruction includes at least one of a matrix multiplication operation, a matrix multiplication-addition operation and a convolution operation; the general operation instruction includes at least one of a floating-point multiplication-addition operation, an integer multiplication-addition operation and a transcendental function operation.

Preferably, the tensor processor includes an instruction decoding module, a first computing module and a second computing module. The instruction decoding module is used to parse the tensor operation instruction, generate a selection signal according to the source address of the data to be operated and obtain the data to be operated, divide the data to be operated into multiple groups of operation data, and send the operation code and the multiple groups of operation data to the first computing module or the second computing module according to the selection signal; the first computing module is used to perform in-memory calculations according to the received operation code, multiple groups of operation data and target address; the second computing module is used to perform non-in-memory calculations according to the received operation code, multiple groups of operation data and target address.

Preferably, the instruction decoding module includes: an instruction parsing unit, used to parse the tensor operation instruction to obtain the operation code and the source address and target address of the data to be operated; a data grouping unit, used to obtain the data to be operated according to the source address of the data to be operated, and divide the data to be operated into multiple groups of operation data; a control unit, used to generate a selection signal according to the source address of the data to be operated, and send the operation code, multiple groups of data and the target address to the first computing module or the second computing module according to the selection signal.

Preferably, the first computing module and the second computing module are connected to a general processor using a unified interface.

Preferably, the unified interface includes one of a PCIE interface and a UCIE interface.

Preferably, the general purpose processor and the tensor processor are packaged in the same core.

Preferably, the general-purpose processor and the tensor processor are packaged into different cores and integrated into the same chip or deployed on different chips.

Preferably, the general-purpose processor is any one of a CPU, a GPU, a DSP, and a GPGPU.

According to a second aspect of the present invention, there is provided a computing method performed by a computing system, wherein the computing system comprises a general-purpose processor and a tensor processor, the general-purpose processor comprises an instruction scheduling module and at least one computing unit, and the computing method comprises: the instruction scheduling module of the general-purpose processor identifies computing instructions; when the instruction scheduling module identifies that the computing instruction is a tensor operation instruction, the tensor operation instruction is sent to the tensor processor; the tensor processor performs in-memory computing and/or non-memory computing according to the tensor operation instruction.

Preferably, the calculation method further comprises:

When the instruction scheduling module identifies that the computing instruction is a general computing instruction, the general computing instruction is sent to the computing unit; and the computing unit performs general computing according to the general computing instruction.

Preferably, the computing instruction includes an operation code and an operation field, wherein the operation code includes the instruction type and operation code of the computing instruction, and the operation field includes the source address and target address of the data to be operated.

Preferably, the instruction types include tensor operations, vector operations and scalar operations.

Preferably, when the instruction type in the computing instruction is a tensor operation, the instruction scheduling module identifies the computing instruction as a tensor operation instruction; when the instruction type in the computing instruction is a vector operation or a scalar operation, the instruction scheduling module identifies the computing instruction as a general operation instruction.

Preferably, the tensor processor performs in-memory calculations and/or non-memory calculations according to the tensor operation instructions, including: parsing the tensor operation instructions to obtain the operation code and the source address and target address of the data to be operated; obtaining the data to be operated according to the source address of the data to be operated, and dividing the data to be operated into multiple groups of operation data; generating a selection signal according to the source address of the data to be operated, and performing in-memory calculations and/or non-memory calculations according to the selection signal, the operation code, multiple groups of data and the target address.

According to a third aspect of the present invention, there is provided a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein the computer program implements the above-described method when executed by a processor.

The computing system, method executed by the computing system, and storage medium provided by the present invention combine a general-purpose processor and a tensor processor together, using the general-purpose processor to process general-purpose calculations in a neural network and using the tensor processor to perform in-memory calculations or non-in-memory calculations, which can both ensure versatility and improve computing power, and support complex and changeable computing operators in neural networks.

Furthermore, the computing instructions use a unified instruction format, and one instruction can be used to drive different computing modules in the tensor processor to perform in-memory or non-memory computing, greatly reducing programming complexity.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

FIG1 is a block diagram showing a computing system according to an embodiment of the present invention;

FIG2 is a schematic diagram showing the structure of an instruction decoding module in a tensor processor provided in an embodiment of the present invention;

FIG3 is a block diagram showing a computing system according to another embodiment of the present invention;

FIG4 is a block diagram showing a computing system according to another embodiment of the present invention;

FIG5 is a flow chart showing a calculation method provided by an embodiment of the present invention;

FIG. 6 shows a flow chart of step S530 provided by an embodiment of the present invention.

DETAILED DESCRIPTION

Various embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. In each of the accompanying drawings, the same elements are represented by the same or similar reference numerals. For the sake of clarity, the various parts in the accompanying drawings are not drawn to scale.

The specific implementation of the present invention is further described in detail below in conjunction with the drawings and examples.

As mentioned above, general-purpose CPUs and GPUs have advantages in data-intensive general-purpose computing related to artificial intelligence, but general-purpose computing is difficult to cope with complex and changeable computing operators. In-memory computing generally adopts the AISC (application-specific integrated circuit) mode, which supports very limited operator types, and has poor programming flexibility and versatility, and cannot flexibly match the current development speed of neural network models.

In response to the above technical problems, the basic idea of the present application is to combine a general-purpose processor and a tensor processor, describe the instruction type in the computing instruction to distinguish between tensor operation instructions and general-purpose operation instructions, send the tensor operation instructions to the tensor processor to perform in-memory calculations or non-memory calculations, and send the general-purpose operation instructions to the general-purpose processor to perform general-purpose calculations. This can not only ensure versatility but also improve computing power, and support complex and changeable computing operators in neural networks.

Fig. 1 is a schematic diagram showing the structure of a computing system provided according to an embodiment of the present application. As shown in Fig. 1 , the computing system 100 includes a general purpose processor 110 and a tensor processor 120 .

The general purpose processor 110 includes an instruction scheduling module 111 and at least one computing unit CU.

The instruction scheduling module 111 is used to identify a computing instruction, and when the computing instruction is identified as a tensor operation instruction, send the tensor operation instruction to the tensor processor 120 .

In this embodiment, the computing instruction includes an operation code and an operation field, wherein the operation code includes the instruction type and operation code of the computing instruction, and the operation field includes at least a source address and a target address of data to be operated.

Specifically, the instruction type is used to describe the type of operation involved in the computing instruction, and the instruction type includes tensor operation, vector operation, scalar operation and transcendental function operation. The operation code is used to describe the operation to be completed by the computing instruction (for example, addition, subtraction, multiplication, division, special function, etc.), which specifically describes the nature and function of the operation.

The operation domain includes the source address and target address of the data to be operated. The source address and the target address can be a memory address or a register address (i.e., a register number). The memory or register can be an off-chip memory. Of course, in practical applications, it can also be an on-chip memory for storing data. The data can specifically be 1 data value (scalar) or n-dimensional data (vector or tensor), where n is an integer greater than or equal to 1. For example, when n=1, it is 1-dimensional data, i.e., a vector; when n=2, it is 2-dimensional data, i.e., a matrix; when n=3 or more, it is a multi-dimensional tensor.

In this embodiment, the opcode may be the part of the instruction or field (usually represented by a code) specified in the computer program to perform the operation, which is an instruction sequence number used to inform the device executing the instruction which specific instruction needs to be executed. The operation domain may be the source of all data required to execute the corresponding instruction, such as the corresponding address, etc. All data required to execute the corresponding instruction include the data to be calculated and the corresponding instruction processing method, etc. For a calculation instruction, it must include an opcode and an operation domain, wherein the operation domain at least includes the source address and the target address of the data to be calculated. It should be understood that those skilled in the art can set the instruction format of the calculation instruction and the opcode and operation domain contained therein as needed, and the present disclosure does not limit this.

In a preferred embodiment, the source address of the data to be operated may be the starting address of the storage space where the data to be operated is located, and the general processor 110 or the tensor processor 120 may obtain instructions and data through a data input and output unit, which may be one or more data I/O interfaces or I/O pins. Further, the general processor 110 or the tensor processor 120 may determine the data to be operated according to the source address of the data to be operated, and obtain the data to be operated. Of course, in other embodiments, the general processor 110 or the tensor processor 120 may also determine the data required for the operation according to the operation code of the calculation instruction.

In one possible implementation, the instruction format of the calculation instruction may be as shown in the following table:

Table 1:

Wherein, type, opcode is the operation code of the computing instruction, dst, scr is the operation domain of the computing instruction, and dst is the target address. src is the source address of the data to be operated. When there are multiple data to be operated, src may include multiple data to be operated src0, src1, ..., srcn, and dst may also include multiple target addresses dst0, dst1, ..., dstn, which is not limited in the present disclosure.

In some specific embodiments, the scalar operation and vector operation in the general operation instruction may be shown in the following table:

Table 2:

When executing the operation instructions shown in the above table, the general processor 110 reads Num1 floating-point data or integer data from the address specified in the register Reg1, reads Num2 floating-point data or integer data from the address specified in the register Reg2, then performs multiplication and addition operations, and then stores the calculation results in the address space specified in the register Reg3.

In some specific embodiments, the transcendental function operation in the general operation instruction may be as shown in the following table:

Table 3:

When executing the operation instructions shown in the above table, the general processor 110 reads Num1 data from the address specified in the register Reg1, reads Num2 data from the address specified in the register Reg2, then performs the function operation, and then stores the calculation result in the address space specified in the register Reg3.

In some specific embodiments, the tensor operation instruction may be as shown in the following table:

Table 4:

When executing the operation instructions shown in the above table, the tensor processor 120 reads Num1 tensor data from the address specified in the register Reg1, reads Num2 weight data from the address specified in the register Reg2, and then performs a convolution operation or a multiplication and addition operation, and then stores the calculation result in the address space specified in the register Reg3.

In a preferred embodiment, if there is no information on the number of data that needs to be read in the operation domain of the operation instruction, it means that the data that needs to be read is a fixed number of data, which can be one data, a row of data, or a column of data, etc., and the embodiment of the present application does not make specific restrictions.

In this embodiment, when the instruction type in the computing instruction is a tensor operation (for example, the instruction type is Tensor), the instruction scheduling module 111 identifies the computing instruction as a tensor operation instruction; when the instruction type in the computing instruction is one of a vector operation, a scalar operation, and a transcendental function operation (for example, the instruction type is Float, INT, SUF), the instruction scheduling module 111 identifies the computing instruction as a general operation instruction. The tensor operation instruction includes at least one of a matrix multiplication operation (MM), a matrix multiplication and addition operation (MAC), and a convolution operation (Conv); the general operation instruction includes at least one of a floating-point multiplication and addition operation, an integer multiplication and addition operation, and a transcendental function operation.

The instruction scheduling module 111 is further configured to send the general operation instruction to the computing unit CU when identifying that the computing instruction is a general operation instruction; the computing unit CU is configured to perform general calculation according to the general operation instruction.

FIG1 specifically shows two computing units CU as examples, and omits other possible computing units. Each computing unit CU includes an instruction distribution module, multiple computing cores (Kernel), a register file, a shared L1 cache, etc. The instruction scheduling module 111 is also used to schedule the execution of computing tasks between multiple computing units CU. The general-purpose processor in this embodiment is any one of a CPU, a GPU, a DSP, and a GPGPU.

The computing system can be used for computing tasks such as matrix calculations, which can be executed in parallel by multiple threads. For example, before execution, these threads are divided into multiple thread blocks in the instruction scheduling module 111, and then these thread blocks are distributed to each computing unit CU (for example, a streaming multiprocessor (SM)). All threads in a thread block are usually assigned to the same computing unit for execution. At the same time, the thread block is split into thread bundles (or simply thread bundles, thread warps), for example, each thread bundle contains a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can be executed in the same computing unit, or in different computing units.

In each computing unit, the instruction distribution module 112 schedules and distributes the thread bundles so that the multiple computing cores of the computing unit CU run the corresponding thread bundles. Each computing core includes an arithmetic logic unit (ALU), a floating point computing unit, etc. According to the number of computing cores in the computing unit, multiple thread bundles in a thread block can be executed simultaneously or in time-sharing. Multiple threads in each thread bundle will execute the same instruction, and the result obtained after the instruction is executed is updated to the register corresponding to each thread bundle. The instructions and data corresponding to each computing unit CU are sent to the shared cache (e.g., shared L1 cache) in the computing unit or further sent to the unified cache for read and write operations, etc.

The tensor processor 120 is used to perform in-memory calculations and/or non-in-memory calculations according to the tensor operation instructions.

In this embodiment, the tensor processor 120 includes an instruction decoding module 121 , a first computing module 122 , and a second computing module 123 .

Among them, the instruction decoding module 121 is used to parse the tensor operation instruction, generate a selection signal according to the source address of the data to be operated and obtain the data to be operated, divide the data to be operated into multiple groups of operation data, and send the operation code and multiple groups of operation data to the first computing module 122 or the second computing module 123 according to the selection signal.

In this embodiment, the instruction decoding module 121 obtains the data to be operated and generates a selection signal according to the source address of the data to be operated described in the tensor operation instruction. The data to be operated includes activation data and weight data; if the source address of the weight data is a storage-computation integrated unit, the weight data is static data, and the instruction decoding module 121 sends the operation code and multiple groups of operation data to the first calculation module 122 according to the selection signal; if the source address of the weight data is a memory, the weight data is dynamic data, and the instruction decoding module 121 sends the operation code and multiple groups of operation data to the second calculation module 123 according to the selection signal. In other embodiments, the instruction decoding module 121 can also generate a selection signal according to the operation code.

Referring to Figure 2, the instruction decoding module 121 includes an instruction parsing unit 1211, a data grouping unit 1212 and a control unit 1213, wherein the instruction parsing unit 1211 is used to parse the tensor operation instruction to obtain the operation code and the source address and target address of the data to be operated; the data grouping unit 1212 is used to obtain the data to be operated according to the source address of the data to be operated, and divide the data to be operated into multiple groups of operation data; the control unit 1213 is used to generate a selection signal according to the source address of the data to be operated, and send the operation code, multiple groups of data and the target address to the first computing module 122 or the second computing module 123 according to the selection signal.

In this embodiment, the operation domain may also include an execution amount. The instruction decoding module 121 is also used to obtain the execution amount and divide the data to be operated into multiple groups of operation data according to the execution amount. The execution amount is the amount of data that the first calculation module 122 or the second calculation module 123 can execute and process at one time.

In a preferred embodiment, when the operation domain does not include the execution amount, the data to be operated can be divided into multiple groups of operation data according to a preset default execution amount.

The first calculation module 122 is used to perform in-memory calculation according to the received operation code, multiple sets of operation data and target address.

In this embodiment, the first computing module 122 can perform in-memory computing (CIM). The first computing module 122 is composed of an SRAM, ReRAM or other storage medium storage and computing integrated unit.

The second calculation module 123 is used to perform non-memory calculation according to the received operation code, multiple groups of operation data and target address.

In this embodiment, the non-in-memory calculation includes near-memory calculation or other calculation, such as general matrix multiplication (GEMM for short).

In a preferred embodiment, the first computing module 122 and the second computing module 123 are connected to the general processor 110 using a unified interface, and the unified interface can be one of a PCIE interface and a UCIE interface. In this embodiment, the data structures of the input data and the output data of the first computing module 122 and the second computing module 123 are the same, and the first computing module 122 and the second computing module 123 can be driven by one instruction to perform calculations, and the first computing module 122 or the second computing module 123 can be switched to perform calculations through instructions.

In this embodiment, the general processor 110 and the tensor processor 120 can be packaged in the same core. In a preferred embodiment, referring to FIG3 , the general processor 110 and the tensor processor 120 can also be packaged into different cores, integrated into the same chip or deployed on different chips. The positional relationship between the general processor 110 and the tensor processor 120 can be set according to actual applications, and is not limited thereto.

The computing system provided by the present invention combines a general-purpose processor and a tensor processor together, uses the general-purpose processor to process general-purpose calculations in neural networks and uses the tensor processor to perform in-memory calculations or non-in-memory calculations, which can not only ensure versatility but also improve computing power and support complex and changeable computing operators in neural networks.

Furthermore, the computing instructions use a unified instruction format, and one instruction can be used to drive different computing modules in the tensor processor to perform in-memory or non-memory computing, greatly reducing programming complexity.

FIG4 is a schematic diagram showing the structure of a computing system provided by another embodiment of the present invention. As shown in FIG4 , the computing system includes a task distribution module 310 and a plurality of computing devices 320 .

This embodiment is described by taking two computing devices 320A and 320B as an example, but is not limited thereto.

In this embodiment, the task distribution module 310 is used to distribute the computing instructions to multiple computing devices 320. The computing device 320 receives the computing instruction and performs corresponding calculations according to the computing instruction. The computing devices 320A and 320B include a general processor 321 and a tensor processor 322. The general processor 321 and the tensor processor 322 are the same as those described in the above embodiment, and will not be repeated here.

In other embodiments, the computing devices 320A and 320B only include a general-purpose processor 321, the tensor processor 322 is located outside the computing devices, and multiple computing devices 320 share the same tensor processor.

Fig. 5 shows a flow chart of a calculation method provided by an embodiment of the present invention. Referring to Fig. 5, the calculation method is executed by the calculation system 100 provided by the above embodiment, and includes the following steps.

In step S510 , an instruction dispatching module of a general purpose processor identifies a computing instruction.

In this embodiment, the computing instruction includes an operation code and an operation field, wherein the operation code includes the instruction type and operation code of the computing instruction, and the operation field includes at least a source address and a target address of data to be operated.

Specifically, the instruction type is used to describe the type of operation involved in the computing instruction, and the instruction type includes tensor operation, vector operation, scalar operation and transcendental function operation. The operation code is used to describe the operation to be completed by the computing instruction (for example, addition, subtraction, multiplication, division, special function, etc.), which specifically describes the nature and function of the operation.

The operation domain includes the source address and target address of the data to be operated. The source address and the target address can be a memory address or a register address (i.e., a register number). The memory or register can be an off-chip memory. Of course, in practical applications, it can also be an on-chip memory for storing data. The data can specifically be 1 data value (scalar) or n-dimensional data (vector or tensor), where n is an integer greater than or equal to 1. For example, when n=1, it is 1-dimensional data, i.e., a vector; when n=2, it is 2-dimensional data, i.e., a matrix; when n=3 or more, it is a multi-dimensional tensor.

In this embodiment, when the instruction type in the computing instruction is a tensor operation (for example, the instruction type is Tensor), the instruction scheduling module 111 identifies the computing instruction as a tensor operation instruction; when the instruction type in the computing instruction is one of a vector operation, a scalar operation, and a transcendental function operation (for example, the instruction type is Float, INT, SUF), the instruction scheduling module 111 identifies the computing instruction as a general operation instruction. The tensor operation instruction includes at least one of a matrix multiplication operation (MM), a matrix multiplication and addition operation (MAC), and a convolution operation (Conv); the general operation instruction includes at least one of a floating-point multiplication and addition operation, an integer multiplication and addition operation, and a transcendental function operation.

In step S520, when the instruction scheduling module identifies that the computing instruction is a tensor operation instruction, the tensor operation instruction is sent to the tensor processor.

In step S530, the tensor processor performs in-memory calculations and/or non-in-memory calculations according to the tensor operation instructions.

In this embodiment, referring to FIG. 6 , step S530 includes steps S531 to S533 .

In step S531, the tensor operation instruction is parsed to obtain the operation code and the source address and target address of the data to be operated.

In step S532, the data to be operated is obtained according to the source address of the data to be operated, and the data to be operated is divided into a plurality of groups of operation data.

In this embodiment, the operation domain may also include an execution amount. The instruction decoding module 121 is also used to obtain the execution amount and divide the data to be operated into multiple groups of operation data according to the execution amount. The execution amount is the amount of data that the first calculation module 122 or the second calculation module 123 can execute and process at one time.

In a preferred embodiment, when the operation domain does not include the execution amount, the data to be operated can be divided into multiple groups of operation data according to a preset default execution amount.

In step S533, a selection signal is generated according to the source address of the data to be operated, and in-memory calculation and/or non-in-memory calculation is performed according to the selection signal, the operation code, multiple groups of data and the target address.

In this embodiment, the data to be operated includes activation data and weight data; if the source address of the weight data is a storage-computation integrated unit, the weight data is static data, and the instruction decoding module 121 sends the operation code and multiple groups of operation data to the first calculation module 122 according to the selection signal; if the source address of the weight data is a memory, the weight data is dynamic data, and the instruction decoding module 121 sends the operation code and multiple groups of operation data to the second calculation module 123 according to the selection signal. In other embodiments, the instruction decoding module 121 can also generate a selection signal according to the operation code.

In step S540, when the instruction scheduling module identifies that the computing instruction is a general operation instruction, the general operation instruction is sent to the computing unit.

In step S550, the computing unit performs general computing according to the general computing instruction.

The computing method provided by the present invention combines a general-purpose processor and a tensor processor together, uses the general-purpose processor to process general-purpose calculations in a neural network, and uses the tensor processor to perform in-memory calculations or non-in-memory calculations, which can both ensure versatility and improve computing power, and support complex and changeable computing operators in neural networks.

Furthermore, the computing instructions use a unified instruction format, and one instruction can be used to drive different computing modules in the tensor processor to perform in-memory or non-memory computing, greatly reducing programming complexity.

An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device/electronic device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

In the embodiments provided in the present application, it should be understood that the disclosed devices/electronic devices and methods can be implemented in other ways. For example, the device/electronic device embodiments described above are merely schematic. For example, the block division of the modules or units is only a logical function block division. There may be other block division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

According to the embodiments of the present invention as described above, these embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made based on the above description. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can make good use of the present invention and the modified use based on the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims (21)

A computing system, characterized by comprising a general purpose processor and a tensor processor; The general purpose processor includes an instruction scheduling module and at least one computing unit. The instruction scheduling module is used to identify the computing instruction, and when the computing instruction is identified as a tensor operation instruction, send the tensor operation instruction to the tensor processor; A tensor processor is used to perform in-memory calculations and/or non-memory calculations according to the tensor operation instructions. The computing system according to claim 1, characterized in that the instruction scheduling module is further used to send the general operation instruction to the computing unit when the computing instruction is identified as a general operation instruction; The computing unit is used to perform general computing according to general computing instructions. The computing system according to claim 2 is characterized in that the computing unit includes at least one computing core, and the computing core is used to perform general computing according to general operation instructions. The computing system according to claim 1 is characterized in that the computing instruction includes an operation code and an operation field, wherein the operation code includes an instruction type and an operation code of the computing instruction, and the operation field includes a source address and a target address of the data to be operated. The computing system according to claim 4 is characterized in that the instruction types include tensor operations, vector operations, scalar operations and transcendental function operations. The computing system according to claim 5 is characterized in that when the instruction type in the computing instruction is a tensor operation, the instruction scheduling module recognizes that the computing instruction is a tensor operation instruction; when the instruction type in the computing instruction is one of a vector operation, a scalar operation and a transcendental function operation, the instruction scheduling module recognizes that the computing instruction is a general operation instruction. The computing system according to claim 1 is characterized in that the tensor operation instructions include at least one of matrix multiplication operations, matrix multiplication and addition operations, and convolution operations; and the general operation instructions include at least one of floating-point multiplication and addition operations, integer multiplication and addition operations, and transcendental function operations. The computing system according to claim 4, characterized in that the tensor processor comprises an instruction decoding module, a first computing module and a second computing module, The instruction decoding module is used to parse the tensor operation instruction, generate a selection signal according to the source address of the data to be operated and obtain the data to be operated, divide the data to be operated into multiple groups of operation data, and send the operation code and the multiple groups of operation data to the first computing module or the second computing module according to the selection signal; The first calculation module is used to perform in-memory calculation according to the received operation code, multiple groups of operation data and target address; The second calculation module is used for performing non-memory calculation according to the received operation code, multiple groups of operation data and target address. The computing system according to claim 8, wherein the instruction decoding module comprises: An instruction parsing unit, used for parsing the tensor operation instruction to obtain an operation code and a source address and a target address of data to be operated; A data grouping unit, configured to obtain the data to be operated according to the source address of the data to be operated, and divide the data to be operated into a plurality of groups of operation data; The control unit is used to generate a selection signal according to the source address of the data to be operated, and send the operation code, multiple groups of data and the target address to the first calculation module or the second calculation module according to the selection signal. The computing system according to claim 8 is characterized in that the first computing module and the second computing module are connected to a general-purpose processor using a unified interface. The computing system according to claim 10, characterized in that the unified interface includes one of a PCIE interface and a UCIE interface. The computing system according to claim 1, characterized in that the general-purpose processor and the tensor processor are packaged in the same chip. The computing system according to claim 1 is characterized in that the general-purpose processor and the tensor processor are packaged into different core particles and integrated into the same chip or deployed on different chips. The computing system according to claim 1 is characterized in that the general-purpose processor is any one of a CPU, a GPU, a DSP, and a GPGPU. A computing method performed by a computing system, characterized in that the computing system includes a general-purpose processor and a tensor processor, the general-purpose processor includes an instruction scheduling module and at least one computing unit, and the computing method includes: The instruction dispatch module of the general purpose processor identifies the computational instructions; When the instruction scheduling module identifies that the computing instruction is a tensor operation instruction, the tensor operation instruction is sent to the tensor processor; The tensor processor performs in-memory calculations and/or non-in-memory calculations according to the tensor operation instructions. The calculation method according to claim 15, further comprising: When the instruction scheduling module identifies that the computing instruction is a general computing instruction, the general computing instruction is sent to the computing unit; The computing unit performs general calculations according to general operation instructions. The computing method according to claim 15 is characterized in that the computing instruction includes an operation code and an operation field, wherein the operation code includes an instruction type and an operation code of the computing instruction, and the operation field includes a source address and a target address of the data to be operated. The computing method according to claim 17 is characterized in that the instruction types include tensor operations, vector operations and scalar operations. The computing method according to claim 18 is characterized in that when the instruction type in the computing instruction is a tensor operation, the instruction scheduling module recognizes that the computing instruction is a tensor operation instruction; when the instruction type in the computing instruction is a vector operation or a scalar operation, the instruction scheduling module recognizes that the computing instruction is a general operation instruction. The computing method according to claim 17, characterized in that the tensor processor performs in-memory computing and/or non-memory computing according to the tensor operation instruction comprises: Parsing the tensor operation instruction to obtain an operation code and a source address and a target address of data to be operated; Acquire the data to be operated according to the source address of the data to be operated, and divide the data to be operated into multiple groups of operation data; A selection signal is generated according to the source address of the data to be operated, and in-memory calculation and/or non-in-memory calculation is performed according to the selection signal, the operation code, multiple groups of data and the target address. A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 15 to 20.