WO2025055512A1 - Matrix transposition apparatus and method, ai processor, and computer device - Google Patents

Abstract

The present application belongs to the technical field of artificial intelligence. Disclosed in the embodiments of the present application are a matrix transposition apparatus and method, an AI processor, and a computer device. The matrix transposition apparatus comprises a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer and a controller. The controller is used for: controlling the matrix reader to read from an upper-level memory matrix elements of a matrix to be transposed, the matrix reader reading, for each time, at least one row of matrix elements of the matrix to be transposed; controlling the shifter to execute data shift on the matrix elements read by the matrix reader, and controlling the shifter to write the shifted matrix elements into the matrix buffer; controlling the transposer to read sub-matrixes from the matrix buffer, and controlling the transposer to execute transposition processing on the read sub-matrixes to obtain transposed sub-matrixes, each sub-matrix comprising at least one column of matrix elements of the matrix to be transposed; and controlling the matrix writer to write into a lower-level memory the transposed sub-matrixes output by the transposer.

Description

Matrix transposition device, method, AI processor and computer equipment

This application claims priority to Chinese patent application No. 202311171547.3 filed on September 12, 2023, with invention name “Matrix transposition device, method, AI processor and computer device”, the entire contents of which are incorporated by reference into this application.

Technical Field

The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a matrix transposition device, method, AI processor and computer equipment.

Background Art

With the development of artificial intelligence technology, more and more AI technologies are being used in life and production.

Matrix operations are widely used as the basis of artificial intelligence technology, and matrix transposition is a common matrix operation. For example, when performing image processing, matrix transposition can be used to flip the image; for another example, when performing signal processing, matrix transposition can be used to implement fast Fourier transform.

In order to realize matrix transposition, a special matrix transposition device needs to be set up in the AI processor. Therefore, how to reduce the design complexity of the matrix transposition device and reduce the on-chip circuit area becomes the key to improving the performance of the AI processor.

Summary of the invention

The embodiments of the present application provide a matrix transposition device, method, AI processor and computer equipment, which can reduce the processing delay of matrix transposition operation. The technical solution is as follows:

On the one hand, an embodiment of the present application provides a matrix transposition device, the device comprising:

Matrix readers, shifters, matrix buffers, transposers, matrix writers, and controllers;

The controller is used to control the matrix reader to read the matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time;

The controller is used to control the shifter to perform data shift on the matrix elements read by the matrix reader, and control the shifter to write the shifted matrix elements into the matrix buffer;

The controller is used to control the transposer to read a submatrix from the matrix buffer, and control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed;

The controller is used to control the matrix writer to write the transposed sub-matrix output by the transposer into a lower-level memory.

On the other hand, an embodiment of the present application provides a matrix transposition method, which is used in the matrix transposition device as described in the above aspect, and the method includes:

The controller controls the matrix reader to read the matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time;

The controller controls the shifter to perform data shift on the matrix elements read by the matrix reader, and controls the shifter to write the shifted matrix elements into the matrix buffer;

The controller controls the transposer to read a submatrix from the matrix buffer, and controls the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed;

The controller controls the matrix writer to write the transposed submatrix output by the transposer into a lower-level memory.

On the other hand, an embodiment of the present application provides an AI processor, which includes the matrix transposition device as described in the above aspects.

On the other hand, an embodiment of the present application provides a computer device, which includes an AI processor and a memory as described in the above aspects, and the AI processor is connected to the memory via a bus.

In the embodiment of the present application, the matrix transposition device performs data shift processing on the matrix elements read by the matrix reader. The method comprises the following steps: performing block reading on the matrix stored in the matrix buffer after the shift, obtaining a submatrix including at least one column of matrix elements in the matrix to be transposed, and performing transposition on the submatrix, and then obtaining the transposed result of the matrix to be transposed based on the transposed submatrix corresponding to each submatrix, thereby realizing the matrix transposition along the matrix column direction, which can reduce the processing delay of the matrix transposition compared with performing the matrix transposition along the matrix diagonal direction; in addition, when performing the matrix transposition along the matrix column direction, since the number of matrix elements extracted from the matrix buffer each time is the same, only one transposition logic is needed for the same matrix, which helps to reduce the design complexity of the matrix transposition device and reduce the on-chip area occupied by the matrix transposition device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of an exemplary embodiment of the present application showing a process of performing matrix transposition along a matrix diagonal direction;

FIG2 shows a schematic structural diagram of a matrix transposition device provided by an exemplary embodiment of the present application;

FIG3 is a schematic diagram of an implementation of a matrix filling and filling removal process shown in an exemplary embodiment of the present application;

FIG4 is a schematic diagram of an implementation of a matrix element right shift process shown in an exemplary embodiment of the present application;

FIG5 is a schematic diagram of the structure of a matrix buffer shown in an exemplary embodiment of the present application;

FIG6 is a schematic diagram showing a matrix transposition process according to an exemplary embodiment of the present application;

FIG7 is a schematic diagram of an implementation of a matrix transposition process shown in an exemplary embodiment of the present application;

FIG8 is a schematic diagram of an implementation of a matrix transposition process shown in another exemplary embodiment of the present application;

FIG9 is a schematic diagram of an implementation of a matrix transposition process shown in yet another exemplary embodiment of the present application;

FIG10 shows a flow chart of a matrix transposition method provided by an exemplary embodiment of the present application;

FIG. 11 shows a structural block diagram of a computer device provided by an exemplary embodiment of the present application.

DETAILED DESCRIPTION

Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

Since storage and processing are integrated in neural networks, while storage and processing are separated in classical computer structures (realized by memory and arithmetic units, respectively), when using classical computers to run neural networks, it will inevitably be restricted by the separate storage and processing structure, affecting processing efficiency. In order to improve the processing efficiency of neural networks, neural network processors (Neural-Network Processing Units, NPU) (also known as AI processors) came into being. Among them, AI processors simulate neurons and synaptic structures through circuits. Each neuron is abstracted as an excitation function, and the input of the function is determined by the output of the neurons connected to it and the synapses connecting the neurons. In order to express specific knowledge, it is necessary to adjust the values of synapses in artificial neural networks, the topological structure of the network, etc. This process is called "learning". After learning, artificial neural networks can solve specific problems through the acquired knowledge.

The AI processor needs to perform a large number of matrix operations during operation, including matrix multiplication, matrix inversion, matrix transposition, etc. In order to enable the AI processor to implement matrix transposition, in the related art, a matrix transposition device is specially provided in the AI processor.

In a possible matrix transposition method, the matrix transposition device performs matrix transposition along the diagonal direction of the matrix. As shown in FIG1 , when performing matrix transposition, the matrix transposition device extracts a set of vectors (composed of at least one matrix element) perpendicular to the diagonal direction of the matrix to be transposed each time. Schematically, in (a) shown in FIG1 , the extracted vector includes the matrix element (0,0), in (b) shown in FIG1 , the extracted vector includes the matrix elements (0,1) and (1,0), in (c) shown in FIG1 , the extracted vector includes the matrix elements (0,2), (1,1) and (2,0), and in FIG1 In (d) shown, the extracted vector contains matrix elements (0,3), (1,2), (2,1) and (3,0), and so on.

Furthermore, the matrix transposition device performs a mirror inversion on the read vector, and writes the inverted vector into the memory in the same manner as when it was read. Schematically, in (a) shown in FIG1 , the matrix transposition device writes the matrix element (0,0) into the memory, in (b) shown in FIG1 , the matrix transposition device writes the matrix elements (1,0) and (0,1) into the memory, in (c) shown in FIG1 , the matrix transposition device writes the matrix elements (2,0), (1,1) and (0,2) into the memory, in (d) shown in FIG1 , the matrix transposition device writes the matrix elements (3,0), (2,1) (1,2) and (0,3) into the memory, and so on.

However, when performing matrix transposition in the above manner, the delay of matrix processing is greater than the delay of matrix writing, resulting in poor performance. For example, for the 4×4 matrix shown in Figure 1, writing to the memory requires 4 clock cycles (writing row by row, a total of four rows), while reading the vector along the diagonal to perform the flip requires 7 clock cycles to complete the matrix processing. Therefore, in the scenario of performing continuous multi-matrix transposition operations, the startup delay of the above scheme is greater than the matrix access delay; and in the scenario of performing matrix transposition operations one by one, the use of the above scheme will cause matrix access interruption, resulting in reduced data bandwidth.

In addition, since the number of matrix elements taken out along the diagonal each time is different, the matrix transposition of the same size requires the configuration of multiple vector inversion logics, which increases the design complexity of the inversion circuit and further increases the on-chip area occupied by the matrix transposition device.

Different from the above scheme in which matrix transposition is performed along the matrix diagonal direction, the embodiment of the present application provides a matrix transposition device that performs transposition along the matrix column direction. By performing shifting and block processing on the matrix elements, the matrix transposition device performs transposition on the submatrix containing at least one column of matrix elements in the matrix to be transposed each time, and then obtains the transposition result of the matrix to be transposed based on the transposed submatrix corresponding to each submatrix. Compared with performing matrix transposition along the matrix diagonal direction, the matrix transposition device provided by the embodiment of the present application can achieve a delay consistent with matrix access, that is, reduce the processing delay of matrix transposition; and, since the number of matrix elements extracted from the matrix buffer along the matrix column direction each time is consistent, the matrix of the same length and width size only needs to use one transposition logic in the entire transposition process, which helps to reduce the design complexity of the matrix transposition device and reduce the on-chip area occupied by the matrix transposition device.

The matrix transposition device provided in the embodiment of the present application can be applied to the following scenarios:

1. The training and reasoning process of the neural network in the artificial intelligence chip, including the calculation of the back-propagation gradient when training the neural network, the matrix operations of specific layers in the neural network reasoning process, etc.

2. Image processing process: In image processing applications, when performing matrix transposition calculations, such as image flipping or rotation operations, a digital image represented as a matrix can generate a rotated or mirrored image of the digital image through a transposition operation.

3. Signal processing: Use matrix transposition to implement the Fast Fourier Transform (FFT) algorithm.

4. Relationship analysis in graph networks. Social network or other network analysis can use matrix transposition calculation to determine the source of relationships between nodes in the network, or determine the relationship pattern between nodes in the network.

Please refer to FIG2 , which shows a schematic diagram of a matrix transposition device provided by an exemplary embodiment of the present application. The matrix transposition device includes: a matrix reader 210 , a shifter 220 , a matrix buffer 230 , a transposer 240 , a matrix writer 250 and a controller 260 .

The input end of the matrix reader 210 is connected to the upper memory, the output end of the matrix reader 210 is connected to the input end of the shifter 220, the output end of the shifter 220 is connected to the write end of the matrix buffer 230, the read end of the matrix buffer 230 is connected to the input end of the transposer 240, the output end of the transposer 240 is connected to the input end of the matrix writer 250, and the output end of the matrix writer 250 is connected to the lower memory. The controller 260, as the control end of the matrix transposition process, is respectively connected to the matrix reader 210, the shifter 220, the matrix buffer 230, the transposer 240 and the matrix writer 250.

The controller 260 is used to control the matrix reader 210 to read the matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time.

In different application scenarios, the meanings of matrix elements in the matrix to be transposed may be different. The matrix elements may be the color values of pixels in an image under a certain color channel, or may be the eigenvalues of a certain graph node in a graph network. The embodiments of the present application do not limit the specific meanings of the matrix elements in the transposed matrix.

In some embodiments, the matrix reader 210 reads one row of data (i.e., one burst) from the upper memory each time based on the read interface bit width of the upper memory. Since the total bit width of each row of matrix elements in the matrix to be transposed may be less than or equal to the read interface bit width, each row of data read by the matrix reader 210 contains at least one row of matrix elements in the matrix to be transposed. For example, when the total bit width of each row of matrix elements in the matrix to be transposed is equal to the read interface bit width, one row of matrix elements in the matrix to be transposed is read each time; when the total bit width of each row of matrix elements in the matrix to be transposed is 1/2 of the read interface bit width, two rows of matrix elements in the matrix to be transposed are read each time.

In order to avoid bank (memory block) read conflicts when subsequently reading matrix elements (i.e., to avoid multiple threads attempting to access different locations in the same bank at the same time), before writing the read matrix elements into the matrix buffer 230, the controller 260 is used to control the shifter 220 to perform data shifting on the matrix elements read by the matrix reader 210, and control the shifter 220 to write the shifted matrix elements into the matrix buffer 230.

Optionally, the shift operation may be a circular shift operation performed to the right. For example, when the read matrix elements are 00, 01, 02, 03, 04, 05, after performing a 1-bit circular shift operation to the right, the obtained matrix element sequence is 05, 00, 01, 02, 03, 04.

After the shifted matrix elements are written into the matrix buffer, the matrix elements that were originally in the same column in the matrix to be transposed are moved to different columns, that is, the matrix elements that were originally in the same column in the matrix to be transposed are located in different banks, so there will be no bank read conflict when one or more columns of matrix elements in the matrix to be transposed are subsequently read.

In some embodiments, the data shift lengths of the matrix elements read in each row are different, and the difference between the data shift lengths of the matrix elements in two adjacent rows is the same. For example, the data shift lengths of the matrix elements in four rows are 0, 1, 2, and 3 respectively.

In some embodiments, the matrix buffer 230 can be implemented as a static random-access memory (SRAM) or a register. The embodiments of the present application do not limit the specific type of the matrix buffer.

When the matrix to be transposed is completely written into the matrix buffer 230 , the controller 260 controls the transposer 240 to perform matrix transposition.

The controller 260 is used to control the transposer 240 to read a submatrix from the matrix buffer 230, and control the transposer 240 to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed.

In some embodiments, the matrix to be transposed can be divided into several sub-matrices of uniform size in the column direction (may involve matrix filling, which will be introduced in a separate embodiment below). When the controller 260 controls the transposer 240 to read a sub-matrix from the matrix buffer 230, one sub-matrix among the several sub-matrices is read each time.

Since the size of the sub-matrices read each time is consistent, the transposer 240 can use consistent transposition logic to perform transposition processing on each sub-matrix to obtain a transposed sub-matrix.

In an illustrative example, when the size of the matrix to be transposed is 8×8, the matrix to be transposed can be divided into 8 8×1 sub-matrices along the column direction, and the controller 260 controls the transposer 240 to read one sub-matrix each time, and a total of 8 sub-matrices are read. The transposer 240 performs a transposition process on each sub-matrix to obtain a transposed sub-matrix with a size of 1×8.

Furthermore, the controller 260 is used to control the matrix writer 250 to write the transposed sub-matrix output by the transposer 240 into the lower-level memory.

In some embodiments, after the transposer 240 completes a transposition process, the matrix writer 250 writes the matrix elements of the transposed submatrix obtained by transposition into the lower-level memory. After the matrix writer 250 writes all the transposed submatrices into the lower-level memory, the lower-level memory stores the transposed matrix corresponding to the matrix to be transposed.

Taking the example that the size of the matrix to be transposed is 8×8 and the memory requires 8 clock cycles to read and write the matrix, in the related art, when the matrix transposition is performed along the diagonal direction, the matrix transposition requires 15 clock cycles; while when the matrix transposition device provided in the embodiment of the present application is used to perform the matrix transposition along the column direction, the matrix transposition only requires 8 clock cycles. The matrix transposition is more efficient, and the clock cycle required for the matrix transposition is consistent with the read and write delay of the memory, which helps to improve performance and increase transmission bandwidth.

In addition, in the related art, when performing matrix transposition along the diagonal direction, since the number of matrix elements read each time is 1 to 8, 8 types of inversion logic need to be set; while with the solution provided in the embodiment of the present application, since the number of matrix elements in the sub-matrix read each time is 8, only one transposition logic needs to be set for the transposer, and there is no need to process different numbers of matrix elements, which reduces the design complexity of the matrix transposition device and helps to reduce the on-chip area occupied by the matrix transposition device.

To summarize, in the embodiment of the present application, the matrix transposition device performs data shift processing on the matrix elements read by the matrix reader, performs block reading on the matrix stored in the matrix buffer after the shift, obtains a sub-matrix containing at least one column of matrix elements in the matrix to be transposed, and performs transposition on the sub-matrix, and then obtains the transposition result of the matrix to be transposed based on the transposed sub-matrix corresponding to each sub-matrix, thereby realizing matrix transposition along the matrix column direction, which can reduce the processing delay of the matrix transposition compared to performing matrix transposition along the matrix diagonal direction; in addition, when performing matrix transposition along the matrix column direction, since the number of matrix elements extracted from the matrix buffer each time is the same, the same matrix only needs to use one transposition logic, which helps to reduce the design complexity of the matrix transposition device and reduce the on-chip area occupied by the matrix transposition device.

In a method of dividing a matrix to be transposed into several sub-matrices, for a matrix to be transposed of size N×M, the matrix to be transposed is divided into k sub-matrices, and the total bit width of matrix elements in each sub-matrix is greater than or equal to the read interface bit width, that is, the total bit width of matrix elements in the sub-matrix processed by a single transposition is greater than or equal to the read interface bit width, thereby ensuring that there is no transmission bandwidth loss during the matrix transposition process.

Regarding the manner of determining the number of submatrix columns, in a possible implementation, the controller determines the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed.

The matrix parameters of the matrix to be transposed include the number of rows and columns of the matrix to be transposed and the number of bits of each matrix element.

Since the matrix elements in the matrix are usually stored in the memory in the order of rows first and columns later, when the bit width of the row matrix elements of the matrix to be transposed is greater than or equal to the read interface bit width, it can be ensured that the total bit width of the matrix elements in each matrix processing is greater than or equal to the read interface bit width.

In a possible implementation, when the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, the column-row ratio of the matrix to be transposed is rounded up to an integer and determined as the number of submatrix columns.

The row matrix element bit width is determined based on the number of columns of the matrix to be transposed and the number of bits of each matrix element. Schematically, for an N×M matrix to be transposed, the row matrix element bit width=M(the number of columns of the matrix to be transposed)×Element_size(the number of bits of the matrix element).

For example, when the size of the matrix to be transposed is 8×8 and the number of bits of the matrix elements is 128 bits, the bit width of the row matrix elements of the matrix to be transposed is 1024 bits.

When the row matrix element bit width is equal to the read interface bit width, the controller determines k=ceiling[M/N], where ceilling is a rounding-up operation, that is, the matrix to be transposed is divided into N sub-matrices, and the size of each sub-matrix is N×k.

Correspondingly, the total bit width of the matrix elements in each sub-matrix obtained by division is N×k×Element_size. Since k≥M/N, and M×Element_size＝read_port_width (read interface bit width), N×k×Element_size≥read_port_width.

In an illustrative example, when the row matrix element bit width of the matrix to be transposed is 2048 bits and the read interface bit width is 2048 bits, if the size of the matrix to be transposed is 8×8, the number of submatrix columns is determined to be 1, that is, the size of each submatrix is 8×1; if the size of the matrix to be transposed is 4×8, the number of submatrix columns is determined to be 2, that is, the size of each submatrix is 4×2; if the size of the matrix to be transposed is 6×8, the number of submatrix columns is determined to be 2, that is, the size of each submatrix is 6×2.

In another possible implementation, when the read interface bit width is greater than the row matrix element bit width of the matrix to be transposed, the result of rounding up the ratio of the read interface bit width to the column matrix element bit width of the matrix to be transposed is determined as the number of submatrix columns, and the column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element.

The column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element. Schematically, for an N×M matrix to be transposed, the column matrix element bit width=N(the number of columns of the matrix to be transposed)×Element_size(the number of bits of the matrix element).

For example, when the size of the matrix to be transposed is 8×8 and the number of bits of the matrix elements is 128 bits, the bit width of the column matrix elements of the matrix to be transposed is 1024 bits.

When the row matrix element width is equal to the read interface width, the controller determines k=ceilling[read_port_width/(N*Element_size)], that is, the matrix to be transposed is divided into N sub-matrices, and the size of each sub-matrix is N×k.

Correspondingly, the total bit width of the matrix elements in each sub-matrix obtained by division is N×k×Element_size. Since k≥read_port_width/N*Element_size, N×k×Element_size≥read_port_width.

In an illustrative example, when the row matrix element bit width of the matrix to be transposed is 1024 bits and the read interface bit width is 2048 bits, if the size of the matrix to be transposed is 8×8 (the matrix element is 128 bits), the number of submatrix columns is determined to be 2, that is, the size of each submatrix is 8×2.

In order to ensure that the sizes of the sub-matrices are consistent, when N cannot be divided by M, padding processing needs to be performed on the transposed matrix. Accordingly, when the transposed sub-matrix is subsequently written into the lower-level memory, padding removal needs to be performed.

Schematically, as shown in FIG3, for an N×M matrix to be transposed, since M cannot divide N evenly, the matrix reader fills M'-M matrix elements after the read matrix elements, where M'=k*N. Since the size obtained after transposition is M'×N, it is necessary to remove the last M'-M rows of matrix elements of the transposed matrix to restore the M×N matrix.

In a possible implementation, when the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, and the column-to-row ratio is a non-integer greater than 1, the controller controls the matrix reader to perform padding after the read matrix elements, so that the column-to-row ratio of the matrix to be transposed after padding is an integer.

Optionally, the matrix reader fills the read matrix elements with 0 or other preset data.

In a possible implementation, the controller determines the number of padding columns based on the number of rows and columns of the matrix to be transposed, and controls the matrix reader to perform padding after the read matrix elements based on the number of padding columns.

The number of filled columns is k × N - M. For example, when the size of the matrix to be transposed is 6 × 8, the matrix reader needs to fill 4 columns of matrix elements each time it reads a matrix element.

It should be noted that the matrix reader fills in the matrix elements as it reads them, without taking up any additional time.

Correspondingly, the controller controls the matrix writer to perform padding removal on at least one transposed submatrix output by the transposer, and writes the matrix elements in at least one transposed submatrix after padding removal into the lower-level memory. It should be noted that at least one transposed submatrix after padding removal is located at the last position. In other words, the data position in the at least one transposed submatrix is located at the end of one or more rows of the transposed result of the matrix to be transposed after padding. It should be noted that, in some examples, performing padding removal on the transposed submatrix output by the transposer can be implemented by removing part or all of the data in the transposed submatrix.

Since the filled columns become rows after the submatrix is transposed, the matrix writer needs to perform a de-filling operation on at least one transposed submatrix to remove the rows that the filled columns become after the transposition. It should be noted that the de-filling here does not refer to a specific de-filling operation, but means that the matrix writer does not write some or all of the rows in the transposed submatrix to the lower-level memory.

In one possible implementation, the controller controls the matrix writer to perform padding removal on at least one transposed submatrix output by the transposer based on the number of padding columns, wherein the sum of the number of rows of matrix elements to be removed from at least one transposed submatrix is the same as the number of padding columns.

The number of rows of matrix elements to be removed is consistent with the number of padded columns, that is, k×N-M. For example, when the size of the matrix to be transposed is 6×8, in order to ensure that the column-row ratio of the padded matrix to be transposed is an integer, 4 columns of matrix elements need to be padded, and the size of the padded matrix to be transposed is 6×12. As introduced above, the size of each submatrix is 6×2, and the size of the corresponding transposed submatrix is 2×6. The last 4 rows of matrix elements need to be removed to remove the rows that the padded 4 columns of matrix elements become after transposition. The size of each transposed submatrix is 2×6, that is, it includes two rows and six columns. It is necessary to perform padding removal on the two transposed submatrices in the last position to remove all data in the transposed submatrix to achieve the removal of the last 4 rows of matrix elements.

In order to avoid bank read conflicts when reading a sub-matrix from a matrix buffer, in one possible implementation, the controller determines a shift step corresponding to each matrix element read by the matrix reader based on the number of sub-matrix columns of the sub-matrix, and thereby controls the shifter to perform data right shift on the matrix elements read by the matrix reader based on the shift step.

In one possible implementation, for the matrix element read by the matrix reader for the nth time, the controller determines that the shift step length of the matrix element is (n-1)×k, that is, the shift step length of the matrix element read for the first time is 0, the shift step length of the matrix element read for the second time is k, the shift step length of the matrix element read for the third time is 2k, and so on.

After the above shifting process, the matrix elements originally located in the same column of the matrix to be transposed are moved to different columns, thereby avoiding bank read conflicts when reading sub-matrices from the matrix buffer.

Schematically, as shown in Figure 4, when the number of rows of the matrix to be transposed is 5 and the number of submatrix columns of the submatrix is k, the matrix elements in the first row are shifted right by 0 steps and written into the matrix buffer, the matrix elements in the second row are shifted right by k steps and written into the matrix buffer, the matrix elements in the third row are shifted right by 2k steps and written into the matrix buffer, the matrix elements in the fourth row are shifted right by 3k steps and written into the matrix buffer, and the matrix elements in the fifth row are shifted right by 4k steps and written into the matrix buffer.

In one possible design, the matrix buffer is composed of several memory banks.

In an illustrative example, as shown in FIG5 , 2n memory banks are provided in the matrix buffer, and each memory bank is respectively connected to the write bus and the write data interface, connected to the read data interface through the read bus, and connected to the address interface through the address bus. Therefore, the shifter can write data to different memory banks in parallel through the write data interface, and the transposer can read data from different memory banks in parallel through the read data interface.

Regarding the manner of writing shifted matrix elements into a matrix buffer, in one possible implementation, when executing matrix element writing, the controller is used to control the shifter to write the shifted matrix elements into a plurality of memory banks in the matrix buffer; and when executing matrix element reading, the controller is used to control the transposer to read the matrix elements of the sub-matrix in parallel from the plurality of memory banks in the matrix buffer based on the number of sub-matrix columns.

In some embodiments, the controller controls the shifter to write different matrix elements after shifting into different memory banks in the matrix buffer. For example, when a row of 8 matrix elements is read, the shifter writes the 8 matrix elements into 8 memory banks respectively.

Correspondingly, when the subsequent transposer reads a column of matrix elements from the matrix buffer, the transposer reads 8 matrix elements from 8 memory banks respectively.

In one possible implementation, when reading matrix elements in the same submatrix from different memory banks, the controller determines a read address of each memory bank based on the number of columns of the submatrix, wherein the read address of the memory bank is incremented, and the increment of the address increment is the bit width of k matrix elements.

Schematically, when reading the first sub-matrix, the transposer reads the matrix elements from the first memory bank based on the starting address 0, reads the matrix elements from the second memory bank based on the starting address k×Element_size, reads the matrix elements from the third memory bank based on the starting address 2k×Element_size, and so on. When reading the i-th sub-matrix, the transposer reads the matrix elements from the i-th memory bank based on the starting address 0, reads the matrix elements from the i+1-th memory bank based on the starting address k×Element_size, reads the matrix elements from the i+2-th memory bank based on the starting address 2k×Element_size, and so on. It should be noted that after reading the last memory bank storing the matrix elements, the matrix elements will be read starting from the first memory bank.

Since data shifting is performed when writing matrix elements into the matrix buffer, in order to ensure the correctness of the order of matrix elements in the sub-matrix, in a possible implementation, the controller is also used to control the transposer to perform data left shifting processing on the matrix elements of the sub-matrix based on the number of columns of the sub-matrix.

Optionally, the shift step length of the matrix element left shift is (n-1)×k, that is, the matrix element read for the first time is shifted left by 0, the matrix element read for the second time is shifted left by k, the matrix element read for the third time is shifted left by 2k, and so on.

Schematically, as shown in FIG6 , the matrix to be transposed is divided into 5×k sub-matrices. After the matrix to be transposed after the right shift processing is completely written into the matrix buffer, in the first step, the transposer reads the matrix elements in the first sub-matrix from the matrix buffer, and shifts the matrix elements left by 0 steps, and then performs transposition on the first sub-matrix to obtain a k×5 matrix obtained by transposing the matrix elements from the 1st to the kth columns in the matrix to be transposed, and writes the matrix elements of the first k×5 matrix into the lower-level memory.

In the second step, the transposer reads the matrix elements in the second submatrix from the matrix buffer, shifts the matrix elements left by k steps, and then performs transposition on the second submatrix to obtain a k×5 matrix obtained by transposing the matrix elements in the k+1th to 2kth columns in the matrix to be transposed, and writes the matrix elements of the second k×5 matrix into the lower-level memory.

The processes of the third and fourth steps are similar to the second step (the left shift step lengths are 2k and 3k respectively), and will not be repeated in this embodiment.

In the fifth step, the transposer reads the matrix elements in the fifth submatrix from the matrix buffer, shifts the matrix elements to the left by 4k steps, and then transposes the fifth submatrix to obtain a k×5 matrix obtained by transposing the matrix elements in the 4k+1th to 5kth columns in the matrix to be transposed. Since there is padding data in the k×5 matrix, before writing to the lower-level memory, it is necessary to remove the last several columns of the fifth k×5 matrix based on the number of padding columns when reading, and write the remaining matrix elements into the lower-level memory.

Since the number of matrix elements read from the matrix buffer each time is the same, the same matrix to be transposed can use a transposer with unified transposition logic. In a possible design, the transposer can be composed of multiple multiplexers. When performing transposition processing on the matrix elements, each multiplexer only needs to perform a gated output according to the configured data output terminal.

In one possible implementation, the controller is used to configure the data output end of the multiplexer in the transposer based on the matrix size of the matrix to be transposed, and the transposer is used to perform a gated output on the matrix elements of the submatrix through the configured multiplexer to obtain a transposed submatrix.

The controller configures different multiplexer gating logics for matrices to be transposed of different matrix sizes.

For example, for a matrix to be transposed of type A matrix size, the controller configures the multiplexer selection logic for it so that data input terminal 1 selects data output terminal 1, data input terminal 2 selects data output terminal 2, and data input terminal 3 selects data output terminal 3; for a matrix to be transposed of type B matrix size, the controller configures the multiplexer selection logic for it so that data input terminal 1 selects data output terminal 1, data input terminal 2 selects data output terminal 3, data input terminal 3 selects data output terminal 5, data input terminal 4 selects data output terminal 7, data input terminal 5 selects data output terminal 2, data input terminal 6 selects data output terminal 4, data input terminal 7 selects data output terminal 6, and data input terminal 8 selects data output terminal 8.

Exemplarily, since the number of matrix elements in each transposition sub-matrix is related to the read interface bit width of the upper-level memory, the number of multiplexers in the transposer matches the read interface bit width of the upper-level memory.

Furthermore, in some embodiments, the number of multiplexers is positively correlated with the read interface bit width. For example, when the number of bits of the matrix element is 32 bits, if the read interface bit width is 1024 bits, 32 multiplexers are configured in the transposer, and if the read interface bit width is 2048 bits, 64 multiplexers are configured in the transposer.

In combination with the above embodiments, several typical matrix transposition processes are described below using illustrative examples.

1. The number of rows and columns of the matrix to be transposed is equal, and the bit width of the row matrix element is the same as the read interface bit width of the upper-level memory.

Taking an 8×8 matrix as an example, as shown in FIG7 , the matrix reader reads a row of matrix elements in the matrix to be transposed each time, and right-shifts the read matrix elements with a unit step size of k=1, that is, the matrix elements in the first row are right-shifted by 0, the matrix elements in the second row are right-shifted by 1, the matrix elements in the third row are right-shifted by 2, and so on, thereby writing the matrix to be transposed after the data shift is written into the matrix buffer.

The transposer reads the matrix elements in the first submatrix from the matrix buffer, performs a left shift of 0 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed submatrix obtained by transposition into the lower-level memory.

Similarly, the transposer reads the matrix elements in the second submatrix from the matrix buffer, performs a left shift of 1 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed submatrix obtained by transposition into the lower-level memory.

Similarly, the transposer completes the transposition of the remaining 6 sub-matrices and finally writes the transposed 8×8 matrix into the lower-level cache.

2. The number of rows and columns of the matrix to be transposed is equal, but the bit width of the row matrix element is smaller than the read interface bit width of the upper-level memory.

Taking an 8×8 matrix with a row matrix element width of 1024 bits and a read interface width of 2048 bits as an example, as shown in FIG8 , the matrix reader reads 2 rows of matrix elements in the matrix to be transposed each time, with a unit step of k=2, and performs a right shift on the read matrix elements, that is, the first row of matrix elements is right shifted by 0, the second row of matrix elements is right shifted by 2, the third row of matrix elements is right shifted by 4, and so on, thereby writing the matrix to be transposed after the data shift is written into the matrix buffer.

The transposer reads the matrix elements of the first submatrix from the matrix buffer and performs a left shift of 0 on the matrix elements. The matrix elements are transposed by the configured transposition logic, and finally the matrix elements of the transposed submatrix obtained by transposition are written into the lower-level memory.

Similarly, the transposer reads the matrix elements in the second sub-matrix from the matrix buffer, performs a left shift of 2 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed sub-matrix obtained by transposition into the lower-level memory.

Similarly, the transposer completes the transposition of the remaining two sub-matrices and finally writes the transposed 8×8 matrix into the lower-level cache.

3. The number of rows and columns of the matrix to be transposed is not equal (M is an integer multiple of N), and the bit width of the row matrix element is equal to the read interface bit width of the upper-level memory.

Taking a 4×8 matrix as an example, as shown in FIG9 , the matrix reader reads a row of matrix elements in the matrix to be transposed each time, and right-shifts the read matrix elements with a unit step size of k=2, that is, the matrix elements in the first row are right-shifted by 0, the matrix elements in the second row are right-shifted by 2, the matrix elements in the third row are right-shifted by 4, and so on, thereby writing the matrix to be transposed after the data shift is written into the matrix buffer.

The transposer reads the matrix elements in the first submatrix from the matrix buffer, performs a left shift of 0 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed submatrix obtained by transposition into the lower-level memory.

Similarly, the transposer reads the matrix elements in the second sub-matrix from the matrix buffer, performs a left shift of 2 on the matrix elements, and then performs a transposition process on the matrix elements through the configured transposition logic, and finally writes the matrix elements of the transposed sub-matrix obtained by transposition into the lower-level memory.

Similarly, the transposer completes the transposition of the remaining two sub-matrices and finally writes the transposed 8×4 matrix into the lower-level cache.

It should be noted that when M cannot divide N, additional padding operations need to be performed before data shifting, and padding removal needs to be performed before the last transposed submatrix is written. For example, taking a 4×7 matrix as an example, an additional column of matrix elements needs to be filled before data shifting, and the last row of the transposed submatrix needs to be removed before the last transposed submatrix is written.

In an application scenario of the present application, a matrix transposition device is used to process the training and reasoning process of a neural network, and the matrix transposition device is called a parameter matrix transposition device of a neural network; the device includes: a matrix reader, a shifter, a matrix buffer, a transposition device, a matrix writer, and a controller;

The controller is used to control the matrix reader to read parameter elements of the neural network parameter matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of parameter elements in the neural network parameter matrix each time;

The controller is used to control the shifter to perform data shift on the parameter elements read by the matrix reader, and to control the shifter to write the shifted parameter elements into the matrix buffer;

Optionally, the controller is further used to determine the number of submatrix columns of the submatrix based on the read interface bit width of the upper memory and the matrix parameters of the neural network parameter matrix;

Based on the number of submatrix columns, determining the shift step corresponding to the parameter elements read by the matrix reader each time;

Based on the shift step size, the control shifter performs data right shift on the parameter elements read by the matrix reader.

Optionally, the matrix buffer is composed of a number of memory banks;

The controller is used for controlling the shifter to write the shifted parameter elements into a plurality of memory banks in the matrix buffer;

The controller is further used to control the transposer to read the parameter elements of the sub-matrix in parallel from the plurality of memory banks of the matrix buffer based on the number of columns of the sub-matrix.

The controller is used to control the transposer to read a submatrix from the matrix buffer, and to control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of parameter elements in the neural network parameter matrix;

Optionally, the transposer is composed of a plurality of multiplexers;

The controller is also used to configure the data input of the multiplexer in the transposer based on the matrix size of the neural network parameter matrix. outgoing end;

The transposer is used for performing gated output on matrix elements of the sub-matrix through the configured multiplexer to obtain a transposed sub-matrix.

The controller is used for controlling the matrix writer to write the transposed sub-matrix output by the transposer into the lower-level memory.

The lower-level memory is used to store the transposed result of the neural network parameter matrix, and/or to provide the transposed result of the neural network parameter matrix to a processor (or processing device) that performs the training and reasoning process of the neural network.

In another application scenario of the present application, a matrix transposition device is used to perform image processing, such as realizing image flipping or rotation through matrix transposition, and the matrix transposition device is called an image transposition device; the device includes: a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer, and a controller;

The controller is used to control the matrix reader to read the pixel elements of the image pixel matrix to be transposed from the upper memory, wherein the matrix reader reads at least one row of pixel elements in the image pixel matrix each time;

The controller is used to control the shifter to perform data shift on the pixel elements read by the matrix reader, and to control the shifter to write the shifted pixel elements into the matrix buffer;

Optionally, the controller is further used to determine the number of submatrix columns of the submatrix based on the read interface bit width of the upper memory and the matrix parameters of the image pixel matrix;

Based on the number of submatrix columns, determining the shift step length corresponding to the pixel elements read by the matrix reader each time;

Based on the shift step size, the shifter is controlled to perform data right shift on the pixel elements read by the matrix reader.

Optionally, the matrix buffer is composed of a number of memory banks;

The controller is used for controlling the shifter to write the shifted pixel elements into a plurality of memory banks in the matrix buffer;

The controller is further used for controlling the transposer to read pixel elements of the submatrix in parallel from a plurality of memory banks of the matrix buffer based on the number of columns of the submatrix.

The controller is used to control the transposer to read a submatrix from the matrix buffer, and to control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of pixel elements in the image pixel matrix;

Optionally, the transposer is composed of a plurality of multiplexers;

The controller is also used to configure the data output terminal of the multiplexer in the transposer based on the matrix size of the image pixel matrix;

The transposer is used for performing gate output on the pixel elements of the sub-matrix through the configured multiplexer to obtain a transposed sub-matrix.

The controller is used for controlling the matrix writer to write the transposed sub-matrix output by the transposer into the lower-level memory.

The lower-level memory is used to store the transposed result of the image pixel matrix, and/or to provide the transposed result of the image pixel matrix to a processor (or processing device) that performs the training and reasoning process of the neural network.

In another application scenario of the present application, a matrix transposition device is used to perform signal processing, such as realizing a Fourier transform (FFT) of a signal or a sub-process of the Fourier transform through matrix transposition; the matrix transposition device is called a signal transposition device; the device includes: a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer, and a controller;

The controller is used to control the matrix reader to read the signal elements of the signal matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of signal elements in the signal matrix each time;

The controller is used to control the shifter to perform data shift on the signal elements read by the matrix reader, and to control the shifter to write the shifted signal elements into the matrix buffer;

Optionally, the controller is further used to determine the number of sub-matrix columns of the sub-matrix based on the read interface bit width of the upper-level memory and the matrix parameters of the signal matrix;

Based on the number of submatrix columns, determining the shift step length corresponding to the signal element read by the matrix reader each time;

Based on the shift step size, the shifter is controlled to perform data right shift on the signal elements read by the matrix reader.

Optionally, the matrix buffer is composed of a number of memory banks;

The controller is used for controlling the shifter to write the shifted signal elements into a plurality of memory banks in the matrix buffer;

The controller is also used to control the transposer to read in parallel from multiple memory banks of the matrix buffer based on the number of sub-matrix columns. The signal element of the submatrix.

The controller is used to control the transposer to read a sub-matrix from the matrix buffer, and to control the transposer to perform a transposition process on the read sub-matrix to obtain a transposed sub-matrix, wherein the sub-matrix includes at least one column of signal elements in the signal matrix;

Optionally, the transposer is composed of a plurality of multiplexers;

The controller is further used to configure the data output terminals of the multiplexer in the transposer based on the matrix size of the signal matrix;

The transposer is used to perform gating output on the signal elements of the sub-matrix through the configured multiplexer to obtain a transposed sub-matrix.

The controller is used for controlling the matrix writer to write the transposed sub-matrix output by the transposer into the lower-level memory.

The lower-level memory is used to store the transposed result of the signal matrix, and/or to provide the transposed result of the signal matrix to a processor (or processing device) that performs the training and reasoning process of the neural network.

In another application scenario of the present application, a matrix transposition device is used to perform correlation analysis of a graph network, such as realizing the relationship source between nodes in a calculation network or determining the relationship pattern between nodes in a network through matrix transposition; the matrix transposition device is called an information transposition device; the device includes: a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer, and a controller;

The controller is used to control the matrix reader to read the information elements of the graph network information matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of information elements in the graph network information matrix each time;

The controller is used to control the shifter to perform data shift on the information elements read by the matrix reader, and to control the shifter to write the shifted information elements into the matrix buffer;

Optionally, the controller is further used to determine the number of sub-matrix columns of the sub-matrix based on the read interface bit width of the upper-level memory and the matrix parameters of the graph network information matrix;

Based on the number of submatrix columns, determining the shift step length corresponding to the information elements read by the matrix reader each time;

Based on the shift step length, the shifter is controlled to perform data right shift on the information elements read by the matrix reader.

Optionally, the matrix buffer is composed of a number of memory banks;

The controller is used for controlling the shifter to write the shifted information elements into a plurality of memory banks in the matrix buffer;

The controller is further used to control the transposer to read information elements of the submatrix in parallel from a plurality of memory banks of the matrix buffer based on the number of columns of the submatrix.

The controller is used to control the transposer to read a submatrix from the matrix buffer, and to control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of information elements in the graph network information matrix;

Optionally, the transposer is composed of a plurality of multiplexers;

The controller is also used to configure the data output terminal of the multiplexer in the transposer based on the matrix size of the graph network information matrix;

The transposer is used to perform gating output on the information elements of the sub-matrix through the configured multiplexer to obtain a transposed sub-matrix.

The controller is used for controlling the matrix writer to write the transposed sub-matrix output by the transposer into the lower-level memory.

Among them, the lower-level memory is used to store the transposed result of the graph network information matrix, and/or provide the transposed result of the graph network information matrix to the processor (or processing device) that executes the training and reasoning process of the neural network.

Please refer to FIG. 10 , which shows a flow chart of a matrix transposition method provided by an exemplary embodiment of the present application. The method is applied to a matrix transposition device, and the matrix transposition device includes a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer, and a controller. The method includes:

Step 1001: A controller controls a matrix reader to read matrix elements of a matrix to be transposed from a superior memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time.

In step 1002 , the controller controls the shifter to perform data shift on the matrix elements read by the matrix reader, and controls the shifter to write the shifted matrix elements into the matrix buffer.

Step 1003: The controller controls the transposer to read the submatrix from the matrix buffer, and controls the transposer to read the submatrix from the matrix buffer. The submatrix of the matrix to be transposed is transposed to obtain a transposed submatrix, wherein the submatrix contains at least one column of matrix elements in the matrix to be transposed.

Step 1004: the controller controls the matrix writer to write the transposed sub-matrix output by the transposer into the lower-level memory.

In some embodiments, the controller controls the shifter to perform data shift on the matrix elements read by the matrix reader, including:

Determine the number of submatrix columns of the submatrix based on the read interface bit width of the upper memory and the matrix parameters of the matrix to be transposed;

Based on the number of submatrix columns, determining the shift step length corresponding to the matrix elements read by the matrix reader each time;

Based on the shift step size, the shifter is controlled to perform data right shift on the matrix elements read by the matrix reader.

In some embodiments, determining the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed includes:

When the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, the column-row ratio of the matrix to be transposed is rounded up to determine the number of submatrix columns, and the row matrix element bit width is determined based on the number of columns of the matrix to be transposed and the number of bits of each matrix element;

Or, when the read interface bit width is greater than the row matrix element bit width of the matrix to be transposed, the result of rounding up the ratio of the read interface bit width to the column matrix element bit width of the matrix to be transposed is determined as the number of columns of the submatrix, and the column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element.

In some embodiments, the method further comprises:

When the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, and the column-row ratio is a non-integer greater than 1, the controller controls the matrix reader to perform padding after the read matrix element;

The controller controls the matrix writer to perform padding removal on at least one transposed submatrix output by the transposer, and writes matrix elements in at least one transposed submatrix after padding removal into the lower-level memory.

In some embodiments, the controller controls the matrix reader to perform padding after the read matrix element, including:

Determine the number of padding columns based on the number of rows and columns of the matrix to be transposed;

Control the matrix reader to perform padding after the read matrix elements based on the number of padding columns;

The controller controls the matrix writer to perform padding removal on at least one transposed sub-matrix output by the transposer, including:

The controller controls the matrix writer to perform padding removal on the at least one transposed submatrix output by the transposer based on the number of padding columns, wherein the number of rows of the sum of matrix elements to be removed from the at least one transposed submatrix is the same as the number of padding columns.

In some embodiments, the matrix buffer is composed of a number of memory banks;

The controller controls the transposer to read the sub-matrix from the matrix buffer, including:

Control the shifter to write the shifted matrix elements into multiple memory banks in the matrix buffer;

Based on the number of sub-matrix columns, the transposer is controlled to read the matrix elements of the sub-matrix in parallel from multiple memory banks of the matrix buffer.

In some embodiments, controlling the transposer to read matrix elements of a sub-matrix in parallel from a plurality of memory banks of a matrix buffer comprises:

Determine the read address of each memory bank based on the number of sub-matrix columns, and the read address of the memory bank is incremented;

The transposer is controlled to read the matrix elements of the sub-matrix in parallel from the plurality of memory banks of the matrix buffer based on the read address.

In some embodiments, after the controller controls the transposer to read the sub-matrix from the matrix buffer, the method further includes:

Based on the number of columns of the submatrix, the control transposer performs a data left shift on the matrix elements of the submatrix.

In some embodiments, the transposer is comprised of a plurality of multiplexers;

The method further includes:

The controller configures the data output terminal of the multiplexer in the transposer based on the matrix size of the matrix to be transposed;

The transposer performs gated output on matrix elements of the sub-matrix through a configured multiplexer to obtain a transposed sub-matrix.

In some embodiments, the number of multiplexers in the transposer matches the read interface bit width of the upper level memory.

The detailed process of the matrix transposition device performing matrix transposition can refer to the above-mentioned device embodiment, and this embodiment will not be described in detail here.

In some embodiments, the matrix transposition device in the embodiments of the present application can be integrated into an AI processor.

Please refer to FIG. 11 , which shows a block diagram of a computer device 1100 provided by an exemplary embodiment of the present application. The computer device 1100 may be a portable mobile terminal, such as a smart phone, a tablet computer, a Moving Picture Experts Group Audio Layer III (MP3) player, or a Moving Picture Experts Group Audio Layer IV (MP4) player. The computer device 1100 may also be called a user device, a portable terminal, a workstation, a server, or other names.

Typically, the computer device 1100 includes a processor 1101 and a memory 1102 .

The processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1101 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor for performing processing on data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for performing processing on data in the standby state. In some embodiments, the processor 1101 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

In some embodiments, the processor 1101 may be integrated with the matrix transposition device provided in the above embodiments. When the processor 1101 has a matrix transposition requirement, the matrix transposition device may be used to perform the transposition operation.

The memory 1102 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices, flash memory storage devices.

In some embodiments, the computer device 1100 may also optionally include a peripheral device interface 1103 and at least one peripheral device.

Those skilled in the art will appreciate that the structure shown in FIG. 11 does not limit the computer device 1100 , and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims (19)

A matrix transposition device, characterized in that the device comprises: Matrix readers, shifters, matrix buffers, transposers, matrix writers, and controllers; The controller is used to control the matrix reader to read the matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time; The controller is used to control the shifter to perform data shift on the matrix elements read by the matrix reader, and control the shifter to write the shifted matrix elements into the matrix buffer; The controller is used to control the transposer to read a submatrix from the matrix buffer, and control the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed; The controller is used to control the matrix writer to write the transposed sub-matrix output by the transposer into a lower-level memory. The device according to claim 1, characterized in that the controller is used to: Determine the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed; Determining the shift step length corresponding to the matrix elements read by the matrix reader each time based on the number of columns of the submatrix; Based on the shift step length, the shifter is controlled to perform data right shift on the matrix elements read by the matrix reader. The device according to claim 2, characterized in that the controller is used to: In a case where the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, a result of rounding up the column-row ratio of the matrix to be transposed is determined as the number of columns of the submatrix, and the row matrix element bit width is determined based on the number of columns of the matrix to be transposed and the number of bits of each matrix element; Or, when the read interface bit width is greater than the row matrix element bit width of the matrix to be transposed, the result of rounding up the ratio of the read interface bit width to the column matrix element bit width of the matrix to be transposed is determined as the number of columns of the submatrix, and the column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element. The device according to claim 3, characterized in that the controller is further used for: When the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, and the column-to-row ratio is a non-integer greater than 1, controlling the matrix reader to perform padding after the read matrix element; The matrix writer is controlled to perform padding removal on at least one transposed submatrix output by the transposer, and matrix elements in at least one transposed submatrix after padding removal are written into the lower-level memory. The device according to claim 4, characterized in that the controller is used to: Determine the number of filled columns based on the number of rows and columns of the matrix to be transposed; Controlling the matrix reader to perform padding after the read matrix elements based on the number of padding columns; The controller is further used for: The matrix writer is controlled to perform padding removal on the at least one transposed submatrix output by the transposer based on the padding column number, wherein the sum of the number of rows of matrix elements to be removed from the at least one transposed submatrix is the same as the padding column number. The device according to any one of claims 2 to 5, characterized in that the matrix buffer is composed of a plurality of memory banks; The controller is used to control the shifter to write the shifted matrix elements into a plurality of memory banks in the matrix buffer; The controller is further configured to control the transposer to read matrix elements of the submatrix in parallel from a plurality of memory banks of the matrix buffer based on the number of columns of the submatrix. The device according to claim 6, characterized in that the controller is used to: Determining a read address of each memory bank based on the number of columns of the sub-matrix, the read address of the memory bank being incremented; The transposer is controlled based on the read address to read matrix elements of the sub-matrix in parallel from a plurality of memory banks of the matrix buffer. The device according to any one of claims 2 to 7, characterized in that the controller is further used for: Based on the number of columns of the sub-matrix, the transposer is controlled to perform data left shift on the matrix elements of the sub-matrix. The device according to any one of claims 1 to 8, characterized in that the transposer is composed of a plurality of multiplexers; The controller is further used to configure the data output end of the multiplexer in the transposer based on the matrix size of the matrix to be transposed; The transposer is used to perform gating output on the matrix elements of the sub-matrix through the configured multiplexer to obtain the transposed sub-matrix. The device according to claim 9 is characterized in that the number of multiplexers in the transposer matches the read interface bit width of the upper-level memory. A matrix transposition method, characterized in that the method is applied to a matrix transposition device, the matrix transposition device includes a matrix reader, a shifter, a matrix buffer, a transposer, a matrix writer and a controller, and the method includes: The controller controls the matrix reader to read matrix elements of the matrix to be transposed from the upper-level memory, wherein the matrix reader reads at least one row of matrix elements in the matrix to be transposed each time; The controller controls the shifter to perform data shift on the matrix elements read by the matrix reader, and controls the shifter to write the shifted matrix elements into the matrix buffer; The controller controls the transposer to read a submatrix from the matrix buffer, and controls the transposer to perform a transposition process on the read submatrix to obtain a transposed submatrix, wherein the submatrix includes at least one column of matrix elements in the matrix to be transposed; The controller controls the matrix writer to write the transposed submatrix output by the transposer into a lower-level memory. The method according to claim 11, characterized in that the controller controls the shifter to perform data shift on the matrix elements read by the matrix reader, comprising: The controller determines the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed; The controller determines the shift step length corresponding to the matrix elements read by the matrix reader each time based on the number of columns of the submatrix; The controller controls the shifter to perform data right shift on the matrix elements read by the matrix reader based on the shift step size. The method according to claim 12, characterized in that the controller determines the number of submatrix columns of the submatrix based on the read interface bit width of the upper-level memory and the matrix parameters of the matrix to be transposed, comprising: In a case where the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, the controller determines a result of rounding up the column-row ratio of the matrix to be transposed as the number of columns of the submatrix, and the row matrix element bit width is determined based on the number of columns of the matrix to be transposed and the number of bits of each matrix element; Or, when the read interface bit width is greater than the row matrix element bit width of the matrix to be transposed, the controller determines the number of columns of the submatrix by rounding up the ratio of the read interface bit width to the column matrix element bit width of the matrix to be transposed, and the column matrix element bit width is determined based on the number of rows of the matrix to be transposed and the number of bits of each matrix element. The method according to claim 13, characterized in that the method further comprises: In the case where the read interface bit width is equal to the row matrix element bit width of the matrix to be transposed, and the column-to-row ratio is a non-integer greater than 1, the controller controls the matrix reader to perform padding after the read matrix element; The controller controls the matrix writer to perform padding removal on at least one transposed submatrix output by the transposer, and writes matrix elements in at least one transposed submatrix after padding removal into the lower-level memory. The method according to claim 14, characterized in that the controller controls the matrix reader to perform padding after the read matrix elements, comprising: The controller determines the number of filling columns based on the number of rows and the number of columns of the matrix to be transposed; The controller controls the matrix reader to perform padding after the read matrix element based on the number of padding columns; The controller controls the matrix writer to perform padding removal on at least one transposed sub-matrix output by the transposer, including: The controller controls the matrix writer to perform padding removal on the at least one transposed submatrix output by the transposer based on the number of padding columns, wherein the number of rows of the sum of matrix elements to be removed from the at least one transposed submatrix is the same as the number of padding columns. The method according to any one of claims 12 to 15, characterized in that the method further comprises: Based on the number of columns of the sub-matrix, the controller controls the transposer to perform data left shift on matrix elements of the sub-matrix. The method according to any one of claims 11 to 16, characterized in that the transposer is composed of a plurality of multiplexers; The method further comprises: The controller configures the data output terminal of the multiplexer in the transposer based on the matrix size of the matrix to be transposed; The transposer performs gating output on the matrix elements of the sub-matrix through the configured multiplexer to obtain the transposed sub-matrix. An AI processor, characterized in that the processor comprises the matrix transposition device according to any one of claims 1 to 10. A computer device, characterized in that the computer device comprises the AI processor as claimed in claim 18 and a memory, and the AI processor is connected to the memory via a bus.