CN118312136A - Computing circuits and AI accelerators - Google Patents

Abstract

The embodiment of the present application provides a computing circuit and an artificial intelligence accelerator, which relate to the field of integrated circuit technology. The computing circuit receives multiple first-block floating-point numbers, calculates the multiple first-block floating-point numbers, and obtains multiple second-block floating-point numbers, which includes: a first computing unit, a conversion unit, and a second computing unit; the first computing unit receives multiple first-block floating-point numbers, calculates the mantissa items of the multiple first-block floating-point numbers, and obtains multiple first intermediate floating-point numbers; the conversion unit normalizes the multiple first intermediate floating-point numbers into multiple normalized mantissa items; the second computing unit calculates the multiple normalized mantissa items to obtain multiple second intermediate floating-point numbers, and normalizes the multiple second intermediate floating-point numbers into multiple second-block floating-point numbers based on the normalized mantissa items. The computing circuit reduces the logic complexity and power consumption of the circuit, effectively reduces the data storage requirements, and solves the problem of insufficient storage space and bandwidth for artificial intelligence services.

Description

Computing circuits and AI accelerators

Technical Field

The present application relates to the field of integrated circuit technology, and in particular to a computing circuit and an artificial intelligence accelerator.

Background technique

An AI (Artificial Intelligence) accelerator is a specially designed hardware accelerator or computer system whose main function is to accelerate artificial intelligence applications, especially artificial neural networks, machine learning, machine vision, and other data-intensive or sensor-driven tasks. Artificial intelligence algorithms involve huge amounts of computation and storage. For example, the normalized exponential function (Softmax function) is one of the common operators in artificial intelligence models. It compresses an arbitrary K-dimensional real vector into another K-dimensional real vector, and makes the value range of each element in the new vector within the interval (0,1), and the sum of all elements is 1. In probability theory, the output of the normalized exponential function can be used to represent a classification probability distribution. The mathematical expression of the normalized exponential function is shown as follows:

However, the currently commonly used normalized exponential function calculation circuit uses floating-point numbers as the circuit. The normalized exponential function calculation process involves a large number of exponentiation and division operations. Traditional circuit designs all use floating-point calculation units, that is, floating-point numbers are used as the input and output of the circuit, which has a large area, high power consumption, and high storage and bandwidth requirements, resulting in challenges such as insufficient computing power, storage space, and bandwidth for artificial intelligence services.

Summary of the invention

In order to solve the above technical problems or at least partially solve the above technical problems, an embodiment of the present application provides a computing circuit and an artificial intelligence accelerator.

In a first aspect, an embodiment of the present application provides a calculation circuit, the calculation circuit receiving a plurality of first blocks of floating-point numbers, calculating the plurality of first blocks of floating-point numbers, and obtaining a plurality of second blocks of floating-point numbers; the plurality of first blocks of floating-point numbers share a first exponent term, and the plurality of second blocks of floating-point numbers share a second exponent term;

The calculation circuit comprises: a first calculation unit, a conversion unit and a second calculation unit; the first calculation unit is coupled to the conversion unit, and the conversion unit is coupled to the second calculation unit;

The first calculation unit receives the plurality of first blocks of floating-point numbers, and calculates the mantissa items of the plurality of first blocks of floating-point numbers to obtain a plurality of first intermediate floating-point numbers;

The conversion unit receives the plurality of first intermediate floating-point numbers, and normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items;

The second calculation unit calculates the multiple normalized mantissa items to obtain multiple second intermediate floating-point numbers, and normalizes the multiple second intermediate floating-point numbers into the multiple second block floating-point numbers based on the normalized mantissa items.

In an optional embodiment, the first calculation unit includes a maximum value extraction subunit, a vector subtraction subunit, and a vector index calculation subunit;

The maximum value extraction subunit receives the plurality of first-block floating-point numbers, obtains the largest mantissa item among the mantissa items of the plurality of first-block floating-point numbers, and uses the largest mantissa item as the first mantissa item;

The vector subtraction subunit calculates the difference between the mantissa item of each of the first block floating-point numbers and the first mantissa item respectively, and uses the difference as the second mantissa item;

The vector exponent calculation subunit constructs a plurality of third-block floating-point numbers based on the first exponent term and the second mantissa term, and performs exponential operation on the third-block floating-point numbers to obtain the plurality of first intermediate floating-point numbers.

In an optional embodiment, the maximum value extraction subunit includes a comparator array, and the comparator array compares the mantissa items of the plurality of first block floating-point numbers to obtain the largest mantissa item among the mantissa items of the plurality of first block floating-point numbers.

In an optional embodiment, the vector subtraction subunit comprises a subtractor array, and the subtractor array calculates the difference between the mantissa term of each of the first block floating-point numbers and the first mantissa term.

In an optional embodiment, the vector exponent calculation subunit includes an exponent calculation module, which concatenates the first exponent term and the second mantissa term, queries the exponential function value corresponding to the concatenation result by a table lookup method, and the exponential function value is the first intermediate floating-point number; or, the exponential calculation module obtains the first intermediate floating-point number by a fitting method.

In an optional embodiment, the conversion unit includes a shared index extraction subunit and a normalization subunit;

The shared exponent extraction subunit receives the plurality of first intermediate floating-point numbers, obtains the largest exponent term among the exponent terms of the first intermediate floating-point numbers, and uses the largest exponent term as the third exponent term;

The normalization subunit normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa terms according to the third exponent term.

In an optional embodiment, the normalization subunit includes a plurality of basic circuits, wherein the basic circuits include an exponent extractor, a mantissa extractor, a subtractor, and a shift circuit;

The exponent extractor obtains the exponent term of the first intermediate floating-point number;

The mantissa extractor obtains the mantissa term of the first intermediate floating-point number;

The subtractor obtains a difference between the exponential term of the first intermediate floating-point number and the third exponential term;

The shift circuit shifts and aligns the mantissa term of the first intermediate floating-point number according to the difference between the exponent term of the first intermediate floating-point number and the third exponent term to obtain the normalized mantissa term.

In an optional embodiment, the second calculation unit includes a summation subunit and a multiplication subunit;

The summing subunit calculates the sum of the normalized mantissa items, and takes the reciprocal of the sum of the normalized mantissa items as an intermediate value;

The multiplication subunit calculates the product of the mantissa term of the intermediate value and the normalized mantissa term, normalizes and aligns the third mantissa term with the exponent term of the intermediate value, and obtains a target mantissa term and a target exponent term, wherein the target mantissa term is the mantissa term of the second block of floating-point numbers, and the target exponent term is a second exponent term shared by the second block of floating-point numbers.

In an optional embodiment, the multiplication subunit includes a multiplier and the normalization subunit;

The multiplier calculates the product of the mantissa term of the intermediate value and the normalized mantissa term to obtain a third mantissa term;

The normalization subunit normalizes and aligns the third mantissa term with the exponent term of the intermediate value to obtain a target mantissa term and a target exponent term.

In a second aspect, an embodiment of the present application provides an artificial intelligence accelerator, which includes the computing circuit described in any embodiment of the present application.

The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

The computing circuit provided in the embodiment of the present application receives a plurality of first blocks of floating-point numbers, calculates the plurality of first blocks of floating-point numbers, and outputs a plurality of second blocks of floating-point numbers. The computing circuit includes a first computing unit, a conversion unit, and a second computing unit. The first computing unit receives a plurality of first blocks of floating-point numbers, calculates the mantissa items of the plurality of first blocks of floating-point numbers, and obtains a plurality of first intermediate floating-point numbers; the conversion unit normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items; the second computing unit calculates the plurality of normalized mantissa items, obtains a plurality of second intermediate floating-point numbers, and normalizes the plurality of second intermediate floating-point numbers into a plurality of second blocks of floating-point numbers based on the normalized mantissa items.

The computing circuit provided in the embodiment of the present application uses block floating-point numbers as the input and output of the circuit. Compared with the traditional circuit using floating-point numbers as input and output, the logic complexity and power consumption are greatly reduced, and the data storage requirements can be effectively reduced, solving the problem of insufficient storage space and insufficient bandwidth for artificial intelligence services such as large models. Moreover, based on the characteristics of block floating-point number shared exponents, the computing circuit supports parallel computing, which can effectively improve the computing processing capability, and is easily compatible with other block floating-point operators in the artificial intelligence model algorithm (such as matrix multiplication, etc.), without the need for numerical conversion.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

FIG. 1( a ) shows a schematic diagram of the structure of a single-precision floating point number;

Figure 1(b) shows a schematic diagram of the structure of Google BFloat16;

Figure 1(c) shows a schematic diagram of the structure of a block floating point number;

FIG2 is a schematic diagram showing the structure of a calculation circuit according to an embodiment of the present application;

FIG3 shows a schematic diagram of the structure of a first computing unit in a computing circuit according to an embodiment of the present application;

FIG4 shows a schematic diagram of the structure of a maximum value extraction subunit in a calculation circuit in an embodiment of the present application;

FIG5 is a schematic diagram showing the structure of a vector subtraction subunit in a calculation circuit according to an embodiment of the present application;

FIG6 shows a schematic diagram of the structure of a vector index calculation subunit in a calculation circuit in an embodiment of the present application;

FIG7 shows a schematic diagram of the structure of a conversion unit in a calculation circuit according to an embodiment of the present application;

FIG8 is a schematic diagram showing the structure of a shared index extraction subunit in a calculation circuit according to an embodiment of the present application;

FIG9 is a schematic diagram showing the structure of a normalization subunit in a calculation circuit according to an embodiment of the present application;

FIG10 is a schematic diagram showing the structure of a second computing unit in a computing circuit according to an embodiment of the present application;

FIG11 is a schematic diagram showing the structure of a summing subunit in a calculation circuit according to an embodiment of the present application;

FIG12 is a schematic diagram showing the structure of a multiplication subunit in a calculation circuit according to an embodiment of the present application;

FIG. 13 is a schematic diagram showing data flow of a computing circuit according to an embodiment of the present application.

Detailed ways

The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and/or" in the specification and claims represents at least one of the connected objects, and the character "/" generally indicates that the objects associated with each other are in an "or" relationship.

For ease of understanding, the technical terms related to this application are explained below.

A floating-point number is a number that uses a special encoding format to express the value of a real number. This real number is obtained by multiplying an integer or fixed-point number (i.e., the mantissa) by an integer power of a base number (usually 2 in computers). In computer systems, traditional floating-point number definitions such as float32 (32-bit floating-point number), float64 (64-bit floating-point number), etc. (refer to the binary floating-point arithmetic standard IEEE-754) are mainly composed of three parts: the sign bit, the exponent term, and the mantissa term. Fixed-point numbers are a method of representing numbers used in computers, and their characteristic is that the decimal point position of the numbers involved in the calculation is fixed. Fixed-point numbers include fixed-point integers and fixed-point decimals. Compared with fixed-point numbers, floating-point numbers have higher expression accuracy and dynamic range.

Block floating point is an innovative floating point number. In recent years, due to its small data storage space and high precision, it has gradually become a hot spot in the acceleration of artificial intelligence model reasoning, especially for bandwidth storage bottleneck businesses such as large models. Figure 1 (a), Figure 1 (b), and Figure 1 (c) respectively show single-precision floating point (float32), Google BFloat16 (Google Brain Floating point, 16-bit brain floating point, the main idea is to provide a 16-bit floating point format, its dynamic range is the same as the standard IEEE-FP32, but the precision is lower), and the block floating point structure diagram. In Figure 1 (a), Figure 1 (b), and Figure 1 (c), a0, a1, a2, and an represent different floating point numbers, the purple rectangle represents the sign bit (sign), the blue rectangle represents the exponent term (exponent), and the yellow rectangle represents the mantissa term (mantissa). As shown in Figure 1 (c), the block floating point number consists of a group of sign bits and mantissa terms, and all data in the group share an exponent term.

Fig. 2 shows a schematic diagram of the structure of a calculation circuit according to an embodiment of the present application. The calculation circuit receives a plurality of first-block floating-point numbers, denoted as z _i =2 ^p ·d _i , wherein p and d _i are both fixed-point values, p represents a shared exponent term (i.e., a first exponent term) of the first-block floating-point numbers z _i , d _i represents a mantissa term of each first-block floating-point number, and i is a positive integer greater than 1, such as 15.

The calculation circuit calculates the multiple first block floating point numbers to obtain multiple second block floating point numbers, recorded as _hi = ^2q · _mi , where q and _mi are both fixed point numbers, q represents the shared exponent term (i.e., the second exponent term) of the second block floating point numbers _z , and _mi represents the mantissa term of each second block floating point number.

The computing circuit of the embodiment of the present application can be applied to an artificial intelligence accelerator. For example, it can be applied in an artificial intelligence accelerator to improve the computational efficiency of probability-based multi-classification methods such as multinomial logistic regression, multinomial linear discriminant analysis, and deep learning algorithms, reduce the power consumption of the accelerator and data storage requirements, and effectively save computing resources and storage resources. As an optional example, in the large model Transformer structure, the normalized exponential function is widely used in the multi-head attention layer, as shown in the following formula, so the computing circuit can be applied to the artificial intelligence accelerator to accelerate the computing efficiency of the multi-head attention layer.

As shown in FIG2 , the calculation circuit includes a first calculation unit H1, a conversion unit H2, and a second calculation unit H3. The first calculation unit H1 is coupled to the conversion unit H2, and the conversion unit H2 is coupled to the second calculation unit H3. In the calculation circuit provided in the embodiment of the present invention, the first calculation unit H1, the conversion unit H2, and the second calculation unit H3 are all hardware circuit units composed of hardware circuits. As for the composition of each hardware circuit unit such as the first calculation unit H1, the conversion unit H2, and the second calculation unit H3, any hardware circuit unit that can realize its corresponding function is acceptable, and no limitation is made here.

The first calculation unit H1 receives a plurality of first blocks of floating-point numbers, calculates the mantissa items of the plurality of first blocks of floating-point numbers, and obtains a plurality of first intermediate floating-point numbers.

The goal of the computing circuit in the embodiment of the present application is to calculate in, represents the calculation result corresponding to the first block of floating-point numbers z _i , and K represents the number of floating-point numbers in the first block. Overflow, in the first calculation unit, it is mathematically equivalent, that is, before performing exponential calculation, the mantissa of each first block of floating-point numbers is processed, and the maximum mantissa is subtracted from it, and the expression is substituted to obtain:

Therefore, the first computing unit H1 receives multiple first block floating-point numbers, determines the maximum value among the mantissa terms of the multiple first block floating-point numbers, calculates the difference between the mantissa term of each first block floating-point number and the maximum value of the mantissa term, uses the difference as an exponent term adjustment term, performs exponent calculation based on the first exponent term and the exponent term adjustment term shared by the first block floating-point numbers, and obtains multiple first intermediate floating-point numbers.

In an optional embodiment, a schematic diagram of the structure of the first calculation unit of the embodiment of the present application is shown in Fig. 3. As shown in Fig. 3, the first calculation unit H1 includes a maximum value extraction subunit Q1, a vector subtraction subunit Q2, and a vector index calculation subunit Q3.

The maximum value extraction subunit Q1 receives multiple first block floating point numbers z _i =2 ^p ·d _i , obtains the mantissa item d _i of each first block floating point number, compares the mantissa items d _i of each first block floating point number, obtains the largest mantissa item among the mantissa items of the multiple first block floating point numbers, and uses the largest mantissa item as the first mantissa item d _max .

In some optional implementation scenarios, the structural schematic diagram of the maximum value extraction subunit Q1 is shown in FIG4 , and the maximum value extraction subunit Q1 is composed of a comparator array, and the comparator array is constructed by organizing multiple comparators in a tree structure. The comparator compares two input values and outputs a larger value. FIG4 shows the maximum value extraction subunit Q1 constructed using fifteen comparators, and the maximum value extraction subunit Q1 receives sixteen first block floating point number mantissa items {d ₀ ,d ₁ ,d ₂ …d ₁₅ } and outputs the mantissa item maximum value d _max .

The vector subtraction subunit Q2 calculates the difference between the mantissa item d _i of each first block of floating-point numbers and the first mantissa item d _max , and uses the difference as the second mantissa item d′ _i .

In some optional implementation scenarios, the structural diagram of the vector subtraction subunit Q2 is shown in FIG5 . The vector subtraction subunit Q2 is composed of a subtractor array ( FIG5 shows a vector subtraction subunit constructed using sixteen subtractors). The subtractor inputs the mantissa term d _i (i=0,1,2…15) of the first block of floating-point numbers at one input end and the first mantissa term d _max at the other input end, performs a subtraction operation on the two input values, and outputs the difference d′ _i =d _i -d _max .

The vector exponent calculation subunit Q3 constructs a plurality of third-block floating-point numbers z′ _i =2 ^p ·d′ _i based on the first exponent term p and the second mantissa term d′ _i , and performs exponential operation on the third-block floating-point numbers to obtain a plurality of first intermediate floating-point numbers. The vector exponential calculation subunit Q3 performs exponential calculation on the input block floating point number and outputs a regular floating point number. Optionally, the vector exponential calculation subunit Q3 can implement the exponential calculation function by a fitting method or a look-up table (LUT).

In some optional implementation scenarios, the structural diagram of the vector exponential calculation subunit Q3 is shown in Figure 6. The vector exponential calculation subunit Q3 includes an exponential calculation module LUT. The exponential calculation module LUT splices the fixed-point number p and the fixed-point number d′ _i as a table lookup index, and queries the exponential function value corresponding to the splicing result through the table lookup method, thereby obtaining the floating-point result x _i of the exponential calculation. Wherein, assuming that the first exponent term p is 4-bit data and the second mantissa term d′ _i is 8-bit data, the result obtained by concatenating the first exponent term and the second mantissa term is 12-bit data. For example, if p = y ₁ y ₂ y ₃ y ₄ , the second mantissa term d′ ₁ = y ₅ y ₆ y ₇ y ₈ y ₉ y ₁₀ y ₁₁ y ₁₂ , p is concatenated with d′ ₁ to obtain a concatenated result of y ₁ y ₂ y ₃ y ₄ y ₅ y ₆ y ₇ y ₈ y ₉ y ₁₀ y ₁₁ y ₁₂ . _{y 1} , y ₂ , …, y ₁₂ respectively represent binary values (0 or 1) of different bit positions.

In other optional embodiments, the vector exponent calculation unit Q3 may be composed of a fitting unit, and the floating point number result x _i of the exponent calculation is obtained through the fitting unit.

In the embodiment of the present application, to prevent Overflow, in the first calculation unit, it is mathematically equivalent. Before performing exponential calculation, the mantissa of each first block of floating-point numbers is subtracted from the maximum mantissa, and the expression z _i is substituted to obtain:

The conversion unit H2 receives a plurality of first intermediate floating-point numbers, and normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items.

After receiving multiple first intermediate floating-point numbers, the conversion unit H2 first obtains the exponent term from the binary code of the first intermediate floating-point number, determines the largest exponent term, and then normalizes the multiple first intermediate floating-point numbers based on the maximum value of the exponent term, and uses the normalized result as the mantissa term of the block floating-point number (i.e., the normalized mantissa term).

In an optional embodiment, as shown in FIG. 7 , the conversion unit H2 includes a shared index extraction subunit U1 and a normalization subunit U2 .

The shared exponent extraction subunit U1 receives a plurality of first intermediate floating-point numbers, obtains the largest exponent term among the exponent terms of the first intermediate floating-point numbers, and uses the largest exponent term as the third exponent term p′.

In some optional implementation scenarios, the structural schematic diagram of the shared exponent extraction subunit U1 is shown in FIG8. The shared exponent extraction subunit includes an exponent extraction unit and a comparator array. The exponent extraction unit extracts the exponent term from the input floating point binary code and inputs it to the comparator array. Optionally, the comparator array in the shared exponent extraction subunit U1 can have the same structure as the comparator array shown in FIG4, which compares the input exponent terms and outputs the maximum value of the exponent terms.

The normalization subunit U2 normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items according to the third exponent item p′. For example, the normalization subunit U2 first separates the sign bit, the exponent item, and the mantissa item of the first intermediate floating-point number from the binary code, and subtracts the maximum value of the exponent item from the exponent item, and then uses this value to shift and align the mantissa item, and splices it with the sign bit to obtain the normalized mantissa item.

In some optional implementation scenarios, the structural diagram of the normalization subunit U2 is shown in Figure 9. The normalization subunit is composed of a plurality of parallel basic circuits G, and the basic circuit G includes an exponent extractor, a mantissa extractor, a subtractor and a shift circuit.

The exponent extractor obtains the exponent term of the first intermediate floating-point number.

The mantissa extractor obtains the mantissa term of the first intermediate floating-point number.

The subtractor obtains a difference between the exponent term of the first intermediate floating-point number and the third exponent term.

The shift circuit shifts and aligns the mantissa term of the first intermediate floating point number according to the difference between the exponent term of the first intermediate floating point number and the third exponent term to obtain a normalized mantissa term d″ _i .

The second calculation unit H3 calculates the multiple normalized mantissa items to obtain multiple second intermediate floating-point numbers, and normalizes the multiple second intermediate floating-point numbers into multiple second block floating-point numbers based on the normalized mantissa items.

In some optional embodiments, as shown in FIG. 10 , the second calculation unit H3 includes a summing subunit W1 and a multiplication subunit W2 .

The summing subunit W1 calculates the sum of the normalized mantissa terms d″ _i , takes the reciprocal of the sum of the normalized mantissa terms as the intermediate value, and then outputs the exponential term p″ and the mantissa term of the intermediate value. The output of the summing subunit W1 is shown in the following formula:

In some optional implementation scenarios, the structural diagram of the summing subunit W1 is shown in FIG11. The summing subunit includes an adder array and a reciprocal module. The adder array is constructed by a plurality of adders through a tree structure, and the adder performs an addition operation on two input values and outputs the sum of the two. The reciprocal module calculates the reciprocal of the input value and outputs its exponential term and mantissa term respectively.

The multiplication subunit W2 calculates the product of the mantissa term s of the intermediate value and the normalized mantissa term d″ _i , normalizes and aligns the product with the exponent term of the intermediate value, and obtains a target mantissa term and a target exponent term, wherein the target mantissa term is the mantissa term of the second block of floating-point numbers, and the target exponent term is the second exponent term shared by the second block of floating-point numbers.

In some optional implementation scenarios, the structural diagram of the multiplication subunit W2 is shown in Figure 12. The multiplication subunit W2 includes a multiplier and a normalization subunit.

The multiplier U calculates the product of the mantissa term of the intermediate value and the normalized mantissa term to obtain a third mantissa term d″′ _i =(d″ _i ·s).

The normalization subunit normalizes and aligns the third mantissa term d"' _i with the exponent term p" of the intermediate value to obtain a target mantissa term d"' _i and a target exponent term p"'.

The target mantissa term d″′ _i and the target exponent term p″′ are the mantissa term _mi and the exponent term q of the block floating point number output by the calculation circuit.

The function implemented by the calculation circuit of the embodiment of the present application is to perform normalized exponential calculation on the block floating point number, and the calculation formula is as follows:

In the above calculation formula, the common factor ^2p′ between the numerator and the denominator is eliminated, and the hardware circuit is simplified.

Figure 13 shows a data flow diagram of a computing circuit of an embodiment of the present application. As shown in Figure 13, the computing circuit includes: a maximum value extraction subunit Q1, a vector subtraction subunit Q2, a vector index calculation subunit Q3, a shared index extraction subunit U1, a normalization subunit U2, a summation subunit W1 and a multiplication subunit W2.

The block floating point number z _i =2 ^p ·d _i (i=0, 1, 2, ... 15) is input to the calculation circuit, where p and d _i are both fixed-point values, p is the shared exponent term of each block floating point number z _i , and the shared exponent term is used as the first exponent term. d _i is the mantissa term of each block floating point number z _i .

The maximum value extraction subunit Q1 compares the mantissa terms d _i of each block floating point number z _i and outputs the maximum mantissa term d _max (such as process (1) in FIG. 13 ).

The vector subtraction subunit Q2 calculates the difference between the mantissa term d _i of each block floating point number z _i and the mantissa term maximum value d _max , and uses the difference as the second mantissa term d′ _i =d _i −d _max (such as process (2) in FIG. 13 ).

The vector exponent calculation subunit Q3 constructs a plurality of block floating point numbers z _i ′=2 ^p ·d _i ′ based on the shared exponent term _p and the second mantissa term d′ _i of each block floating point number z i ′, performs exponential operation on the block floating point number z _i ′=2 ^p ·d _i ′, and obtains a plurality of floating point numbers (Process (3) in Figure 13).

The shared exponent extraction subunit U1 receives multiple floating point numbers x _i , obtains the largest exponent term among the exponent terms of the floating point numbers x _i , and uses the largest exponent term as the third exponent term p′ (such as process (4) in FIG. 13 ).

The normalization subunit U2 normalizes the plurality of floating point numbers x _i into a plurality of normalized mantissa terms d″ _i according to the third exponent term p′ (such as process (5) in FIG. 13 ).

The summing subunit W1 calculates the sum of the normalized mantissa terms d″ _i , takes the reciprocal of the sum of the normalized mantissa terms as the intermediate value, and then outputs the exponential term p″ and the mantissa term s of the intermediate value (such as process (6) in Figure 13).

The multiplication subunit W2 calculates the product of the mantissa term s of the intermediate value and the normalized mantissa term d″ _i , normalizes the product and the exponent term of the intermediate value, and obtains the target mantissa term and the target exponent term. The target mantissa term d″′ _i is the mantissa term of the second block of floating-point numbers, and the target exponent term p″′ is the second exponent term shared by the second block of floating-point numbers (such as process (7) in Figure 13).

The target mantissa term d″′ _i and the target exponent term p″′ are the mantissa term and exponent term of the block floating point number output by the calculation circuit.

The computing circuit of the embodiment of the present application uses block floating-point numbers as input and output, which ensures data accuracy and validity, and is easily compatible with other block floating-point operators in the artificial intelligence model algorithm (such as matrix multiplication, etc.), without the need for numerical conversion. At the same time, each subunit uses fixed-point format for calculation, which is fast, has low overhead, and has low storage and bandwidth requirements. Moreover, the computing circuit supports parallel computing based on the characteristics of block floating-point numbers sharing exponents, which can effectively improve computing processing capabilities.

Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The above is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims (10)

1. A computing circuit, which is characterized in that the computing circuit receives a plurality of first block floating point numbers, calculates the plurality of first block floating point numbers, and obtains a plurality of second block floating point numbers; the plurality of first block floating point numbers share a first exponent item, and the plurality of second block floating point numbers share a second exponent item;

the calculation circuit includes: a first calculation unit, a conversion unit, and a second calculation unit; the first computing unit is coupled with the conversion unit, and the conversion unit is coupled with the second computing unit;

The first calculation unit receives the plurality of first block floating point numbers, calculates mantissa items of the plurality of first block floating point numbers, and obtains a plurality of first intermediate floating point numbers;

the conversion unit receives the plurality of first intermediate floating point numbers and normalizes the plurality of first intermediate floating point numbers into a plurality of normalized mantissa items;

The second calculation unit calculates the plurality of normalized mantissa items to obtain a plurality of second intermediate floating point numbers, and normalizes the plurality of second intermediate floating point numbers into the plurality of second block floating point numbers based on the normalized mantissa items.

2. The computing circuit of claim 1, wherein the first computing unit comprises a maximum value extraction subunit, a vector subtraction subunit, a vector exponent computing subunit;

the maximum value extraction subunit receives the plurality of first block floating point numbers, acquires a maximum mantissa item in mantissa items of the plurality of first block floating point numbers, and takes the maximum mantissa item as a first mantissa item;

The vector subtraction subunit calculates the difference value between the mantissa item of each floating point number of the first block and the first mantissa item respectively, and takes the difference value as a second mantissa item;

And the vector exponent calculation subunit constructs a plurality of third block floating point numbers based on the first exponent item and the second mantissa item, and performs exponent operation on the third block floating point numbers to obtain the first intermediate floating point number.

3. The computing circuit of claim 2, wherein the maximum extraction subunit comprises a comparator array that compares mantissa terms of the first plurality of block floating point numbers to obtain a largest mantissa term of the mantissa terms of the first plurality of block floating point numbers.

4. The computing circuit of claim 2, wherein the vector subtraction subunit comprises a subtractor array that calculates a difference between a mantissa term of each of the first block floating point numbers and the first mantissa term.

5. The computing circuit of claim 2, wherein the vector exponent computing subunit includes an exponent computing module;

The exponent calculation module is used for splicing the first exponent item and the second mantissa item, inquiring an exponent function value corresponding to a splicing result through a table lookup method, wherein the exponent function value is the first intermediate floating point number;

or the exponent calculation module obtains the first intermediate floating point number through a fitting method.

6. The computing circuit of claim 1, wherein the conversion unit comprises a shared exponent extraction subunit and a normalization subunit;

the shared index extraction subunit receives the plurality of first intermediate floating point numbers, acquires the largest index item in the index items of the first intermediate floating point numbers, and takes the largest index item as a third index item;

The normalization subunit normalizes the plurality of first intermediate floating-point numbers to a plurality of normalized mantissa items according to the third exponent term.

7. The computing circuit of claim 6, wherein the normalization subunit comprises a plurality of base circuits including an exponent extractor, a mantissa extractor, a subtractor, and a shift circuit;

the exponent extractor acquires an exponent term of the first intermediate floating point number;

the mantissa extractor obtains a mantissa item of the first intermediate floating point number;

the subtracter acquires the difference value between the exponent item of the first intermediate floating point number and the third exponent item;

And the shift circuit shifts and aligns the mantissa items of the first intermediate floating point number according to the difference value of the exponent items of the first intermediate floating point number and the third exponent item to obtain the normalized mantissa items.

8. The computing circuit of claim 6, wherein the second computing unit comprises a summing subunit and a multiplying subunit;

the summation subunit calculates the sum of the normalized mantissa terms, and takes the reciprocal of the sum of the normalized mantissa terms as an intermediate value;

The multiplication subunit calculates the product of the mantissa term of the intermediate value and the normalized mantissa term, normalizes and aligns the third mantissa term with the mantissa term of the intermediate value to obtain a target mantissa term and a target mantissa term, wherein the target mantissa term is the mantissa term of the second block floating point number, and the target mantissa term is the second mantissa term shared by the second block floating point number.

9. The computing circuit of claim 8, wherein the multiplication subunit comprises a multiplier and the normalization subunit;

The multiplier calculates the product of the mantissa term of the intermediate value and the normalized mantissa term to obtain a third mantissa term;

And the normalization subunit performs normalization alignment on the third mantissa item and the exponent item of the intermediate value to obtain a target mantissa item and a target exponent item.

10. An artificial intelligence accelerator comprising the computing circuit of any of claims 1-9.