WO2024045585A1 - Method for dynamically sharing storage space in parallel processor, and corresponding processor - Google Patents

Abstract

Provided in the present application are a method for dynamically sharing a storage space in a parallel processor, and a corresponding processor. One part of a storage space of a local memory of a processor is used as a local memory, and the other part thereof is used as a cache. A memory access control unit of a processor respectively updates the starting positions of a local memory and a cache in a memory of the processor according to received settings regarding the size of the local memory and the size of the cache; and the cache determines a new data storage position corresponding to each cache block in the memory according to the size of the cache and the starting position of the cache in the memory of the processor, and establishes mapping between each tag storage position and the new data storage position of each cache block. The solution allows a user to dynamically adjust the sizes of storage spaces of a local memory and a cache in a processor, such that the execution performance of the processor for an application program is improved, without increasing the area of a chip and the hardware cost.

Description

Method for dynamic shared memory space in parallel processor and corresponding processor Technical field

The present application relates to storage space management technology in high-performance parallel processors, and in particular to a method and system for sharing storage space between the processor's local memory and cache memory.

Background technique

The statements in this section are merely to provide background information related to the technical solution of the present application to help understand, and they do not necessarily constitute prior art for the technical solution of the present application.

High-performance parallel processors such as general-purpose computing on graphics processing units (GPGPU) generally support local memory and global memory programming models, which can provide users with flexibility and allow them to customize the program according to actual conditions. Application to choose local memory or global memory. Local memory and global memory have their own advantages and disadvantages: local memory is generally closer to the computing core and has faster access speed, but its capacity is smaller; while global memory has a large capacity, but its access speed and latency are relatively small. Difference. In response to the access delay problem of global memory, GPGPU uses caching technology to solve the access bottleneck of global memory. For example, a fast but small-capacity cache memory (Cache, which can also be referred to as cache) is set up between the processor and global memory. Cache of global memory data, such as L1 cache. In order to further improve performance, multi-level cache can also be introduced. For example, after the L1 cache, an L2 cache or even an L3 cache can be added to reduce the access delay problem.

Therefore, in parallel processor architectures such as GPGPU that include multiple computing cores, each computing core often contains a local memory and an L1 cache. Generally speaking, the larger the capacity of each computing core's local memory and L1 cache, the better the performance, but correspondingly, this will cause an increase in the chip area and hardware cost of the processor.

It should be noted that the above content is only used to help understand the technical solution of the present application and is not used as a basis for evaluating the prior art of the present application.

Contents of the invention

The inventor's research found that when using programming languages such as CUDA and OPENCL to write applications on GPGPU processors, users will choose a programming model dominated by local memory or global memory based on actual application needs. In other words, user-written applications do not There will be large capacity requirements for local memory and L1 cache at the same time. Therefore, the inventor designed a solution to dynamically share storage space in parallel processors, so that local memory and L1 cache can share storage space, and users can dynamically allocate the size of local memory or L1 cache according to actual applications. . Such a solution allows the same random access memory (RAM) space to be time-shared by local memory and L1 cache, thereby reducing chip area and hardware costs. Because the storage space in local memory or L1 cache usually accounts for 80% of the area cost, using a dynamic shared storage space solution can save about 40% of the area cost.

According to a first aspect of the embodiment of the present application, a method for dynamically sharing memory space in a parallel processor is provided, which includes: a memory access control unit of the processor determines the size of the local memory and the cache memory according to the received information. Setting the size, respectively updating the starting positions of the local memory and the cache memory in the memory of the processor, wherein part of the storage space of the memory is used as the local memory, and the other part is used as the data storage unit of the cache memory; The memory access control unit of the processor updates the setting of the size of the index field in the memory access address of the cache memory according to the received setting of the cache memory size; and the cache memory provides the setting of the cache memory size according to the memory access control unit. The cache memory size and its starting position in the processor's memory determine the new data storage location corresponding to each cache block in the memory, and establish a relationship between each tag storage location and the new data storage location of each cache block. mapping.

In such an embodiment, the local memory and the L1 cache in the processor share the same memory, but the size of the storage space occupied by the local memory and the L1 cache is not fixed, but can dynamically change with the configuration provided by the user. This allows users to re-adjust the local memory and L1 cache in the processor according to the selected programming model based on local memory or global memory when using programming languages such as CUDA and OPENCL to write applications on GPGPU processors. Storage space size to better improve the processor's execution performance for applications. If the user currently chooses programming based on local memory, the storage space of the local memory in the processor can be appropriately expanded. On the contrary, if the user currently chooses programming based on global memory, the storage space of the L1 cache in the processor can be appropriately expanded. In this way, it not only improves the execution performance of the processor for application programs, but also does not increase the chip area and hardware cost.

In some embodiments, the method may further include, by the memory access control unit of the processor, starting from a preset address in the memory of the processor according to the settings it receives for the local memory size and cache memory size. Divide storage space of corresponding size as local memory, and then connect Then allocate storage space for the cache memory, wherein the starting position of the local memory is the preset address.

In such an embodiment, every time the storage space size is adjusted, the starting position of the local memory does not need to change. It only needs to determine the location of the updated cache memory in the processor's memory based on the newly set local memory space size. the new starting position in . This not only simplifies the storage space management process, but also when the size of the local memory is updated, the data in the shared local memory part before and after the update will not be lost.

In some embodiments, the cache may be a set associative cache, and wherein the size of the index field in the access address of the cache is based on the cache size divided by the cache block size preset for the cache and the number of tags contained in each group.

In some embodiments, the method may further include, in response to receiving a memory access request for the local memory, the memory access control unit of the processor using the updated starting position of the local memory to locate the memory access request to access. The data.

In some embodiments, the method may further include: in response to receiving a memory access request for the global memory, the memory access control unit of the processor maps the address in the memory access request into a memory access address of the cache memory, and It is sent to the cache memory, wherein the memory access address adopts the updated index field size; and the cache memory responds to receiving the memory access request from the memory access control unit, according to the established tag storage location and the new Mapping between data storage locations to locate the data to be accessed by the memory access request.

In some embodiments, in response to receiving a memory access request from the memory access control unit, the cache memory locates the memory access request to access according to the established mapping between the tag storage location and the new data storage location. The data can include:

When a cache hit occurs, the cache block corresponding to the hit tag is determined based on the established mapping between the tag storage location and the new data storage location, and the data to be accessed by the memory access request is extracted from the cache block as the cache block for the memory access request. response to the request;

Perform the following actions on a cache miss:

Allocate a tag storage location for the memory access request to store the tag field in the memory access address of the memory access request, and select one of the multiple cache blocks not bound to the tag from the data storage unit of the cache memory to allocate Give the memory access request;

The tag binding bit of the cache block originally corresponding to the allocated tag storage location is set to indicate that it is not bound to the tag, and then between the tag storage location and the cache block allocated to the memory access request Establish a mapping relationship between the cache block assigned to the memory access request and set the tag binding bit to indicate that it has been bound to the tag; and

The data to be accessed by this memory access request is obtained from the next level memory and stored in the cache block allocated for this memory access request.

In such an embodiment, each tag storage location in the cache is no longer fixedly bound to a certain cache block, but can be dynamically mapped or bound to any cache block, and the tag and cache block The data in does not need to be updated synchronously. When the storage space of the L1 cache changes, for example, when the data storage unit 103 of the L1 cache is moved or allocated to another location in the shared storage space, as long as the starting position of the L1 cache in the shared storage space is given, it can Cache blocks are reallocated from this starting position by re-establishing the mapping between each tag storage location and the new data storage location for each cache block accordingly. In this way, when the storage space of the L1 cache changes or is adjusted according to the user's configuration, the data in the newly allocated cache block can be located or found based on the re-established mapping relationship.

According to the second aspect of the embodiment of the present application, a processor that supports dynamic shared storage space is provided, which includes a memory access control unit, a memory and a cache memory. The cache memory includes a controller, a A tag storage unit, a data storage unit composed of multiple cache blocks, and a mapping unit; wherein a part of the storage space of the memory is used as a local memory, and the other part is used as a data storage unit of the cache memory, where:

The memory access control unit is configured to: update the starting positions of the local memory and the cache memory in the memory of the processor according to the received settings of the local memory size and the cache memory size; and according to the received settings The setting of the cache size updates the setting of the size of the index field in the cache access address;

The controller of the cache memory is configured to: determine the new data storage location corresponding to each cache block in the memory based on the size of the cache memory provided by the memory access control unit and its starting position in the memory of the processor. , and establish a mapping between each tag storage location in the tag storage unit and the new data storage location of each cache block in the mapping unit.

In some embodiments, the memory access control unit may also be configured to: first divide the memory starting from a preset address according to the settings it receives for the local memory size and cache memory size. A storage space of a corresponding size is used as a local memory, and then a storage space is allocated for the cache memory, where the starting position of the local memory is the preset address. In some embodiments, the memory may be implemented in the form of random access memory. The mapping unit in the cache memory can be implemented in the form of a register. Such adoption is sent to The mapping unit implemented in the form of a memory can further reduce the cost and area occupied by the storage of mapping relationships in the cache memory, and improve the speed of parsing the mapping relationship between tags and cache blocks.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the application.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts. In the attached picture:

Figure 1 is a schematic diagram of a structural module of a cache memory according to an embodiment of the present application.

Figure 2 is a schematic diagram of the mapping relationship between tags and cache blocks according to an embodiment of the present application.

FIG. 3 is a flowchart illustrating a method for dynamically sharing memory space in a parallel processor according to an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail through specific embodiments in conjunction with the accompanying drawings. It should be understood that the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

Furthermore, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the present application. However, those skilled in the art will appreciate that the technical solutions of the present application may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other instances, well-known methods, apparatus, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the present application.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software form, or in one or more These functional entities are implemented in hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flowcharts shown in the drawings are only illustrative, and do not necessarily include all contents and operations/steps, nor must they be performed in the order described. For example, some operations/steps can be decomposed, and some operations/steps can be merged or partially merged, so the actual order of execution may change according to the actual situation.

In parallel processors such as GPGPU, memory access requests from multiple threads of different computing cores are sent to the processor's memory access control unit (LSU, Load/Store Unit), which is loaded from local memory or global memory by the LSU. memory to access the data accessed by these memory access requests. In a programming model that uses local memory, local memory is usually accessed in the form of an array. Therefore, the memory access control unit can locate the location in the local memory that each thread's memory access request wants to access based on the starting address of the local memory. specific storage location. This is not the case for global memory (also called main memory). In the programming model using global memory, when the memory access control unit receives a memory access request for a certain global memory address, it first processes the memory access request. The global memory address to be accessed is mapped to the access address of the cache memory (such as L1 cache), and then the corresponding data is checked from the L1 cache to see whether it is already cached in the L1 cache. If the data to be accessed is already cached in the L1 cache ( can be called a "hit"), the data is returned directly from the L1 cache to the processor, otherwise the L1 cache controller extracts the data from the next-level memory (such as global memory), caches it, and returns it to the processor.

The inventor's research found that when using programming languages such as CUDA and OPENCL to write applications on GPGPU processors, users will choose a programming model dominated by local memory or global memory based on actual application needs. In other words, user-written applications do not have large requirements for both local memory and L1 cache. Therefore, embodiments of the present application provide a solution in which the storage space of the same random access memory (RAM) is shared by the local memory and the L1 cache in a parallel processor. Users can dynamically allocate the space size of local memory and L1 cache in the processor through upper-layer software according to actual application requirements. Once the space allocation is set, the local memory will only access the designated space, and the L1 cache can also only access the space designated for it. With different user configurations, the space of L1 cache and local memory will also change. For local memory, as long as the starting address and storage space size of the local memory are given, the memory access control unit can locate any storage location in the local memory. This is not the case for the L1 cache. Once the space size of the L1 cache changes, its access address and index space may change. The existing L1 cache control mechanisms are all aimed at fixed space It is set up with a small L1 cache and cannot support dynamic storage space.

More specifically, a cache memory (hereinafter collectively referred to as the L1 cache) located between the processor and the global memory generally includes a controller, a tag storage unit for saving tags, and a data storage unit for saving data. The data storage space of the L1 cache is evenly divided into multiple cache blocks (also called cache lines) of equal size. The smallest unit of data transfer between the L1 cache and global memory is the cache block size. Each cache block in the data storage unit is provided with a unique corresponding tag storage location in the tag storage unit. Tags corresponding to the data stored in each cache block are stored in the corresponding tag storage location of the cache block. When the data in the cache block is updated, the tag in the tag storage location corresponding to the cache block will also be updated, and the two are replaced synchronously. Although tags are also part of the L1 cache, when the L1 cache size is usually mentioned, it only refers to the maximum total amount of data that the L1 cache can accommodate, that is, the storage space size of the data storage unit of the L1 cache.

Existing cache memories are usually divided into three types: Direct mapped Cache, Full associative Cache and Set associative Cache. In a direct-mapped cache, each main memory block can only be mapped to a cache block at a fixed location in the L1 cache. Even if there are many empty locations in the cache, they cannot be occupied. Therefore, the cache storage space cannot be fully utilized, and if The program happens to repeatedly access different main memory blocks corresponding to the same cache location. Block conflicts often occur and require constant replacement, thus reducing the hit rate. In a fully associative cache, each main memory block can be mapped to any cache block in the L1 cache. When accessing data, the address in the memory access request needs to be compared with the tag of each cache block to determine whether it is a hit. However, Each time it is necessary to go through all comparisons to determine the cache miss, and then start the replacement, which directly affects the data cache efficiency and access efficiency. In the set-associative cache, the storage space is divided into several groups. The direct mapping method is used between each group, while the fully associative mapping method is used between each cache block within the group. Correspondingly, the main memory is also partitioned according to the L1 cache size, and each partition is divided into several groups, each group containing several main memory blocks. In this way, each group in main memory can only be mapped to a specified group in Cache, but each main memory block in a group in main memory can be mapped to any cache block in a specified group in Cache. In other words, the within-group mapping method is fully associative, while the between-group mapping method is direct mapping. The set-associative cache is simpler and more efficient than the fully-associative cache in determining block hits and replacement algorithms. The probability of block conflicts is lower than that of the direct-mapped cache, and its hit rate is between the direct-mapped cache and the fully-associative cache. In such an L1 cache, the search and comparison speed of tags will directly affect the processor's data access efficiency. Therefore, the number of tags in the same group is usually not too many. The more common number of tags is usually 4 or 8. or 16, etc., generally less than 32.

In parallel computing processors, set-associative cache is more commonly used as the L1 cache. In the following, an L1 cache of the set-associative cache type is used as an example. However, it should be understood that the embodiments introduced in this application can also be applied to other types of L1 caches with appropriate adjustments or modifications.

When the L1 cache controller receives a memory access request, it determines whether the corresponding data has been cached in the L1 cache through the memory access address in the memory access request. The memory access address usually includes three parts: tag, index and offset. The offset is used to address a certain data in the cache block, the index is used to locate a certain group in the L1 cache, and the tag is used to compare with the corresponding tag of each cache block contained in the group specified by the index. Determine whether it is a hit. Still taking the L1 cache with a size of 64 bytes and a cache block size of 8 bytes as an example, assume that the L1 cache is divided into two groups, and the storage space size of each group is 32 Bytes, that is, it contains 4 cache blocks. When the cache block size is 8 bytes, its addressing range can be represented by 3 bits, so the lowest 3 bits of the access address can be used to represent the offset field; and the number of groups is 2, so 1 bits can cover all groups, so that 1 bit adjacent to the offset can be used in the access address to represent the index field, and the remaining bits in the access address are label fields, used to match all groups in the group located by the index. The tags corresponding to the cache blocks are compared. The tag field stores part of the address of the global memory to be accessed by the memory access request. In this way, when the example L1 cache receives a memory access request, the corresponding group can be located through the index field of the address in the memory access request, and the label field in the memory access address and the information of each cache block contained in the group are obtained. The tags are compared. If there is a matching tag, it means that the cache is hit. The corresponding data in the cache block corresponding to the matched tag can be extracted according to the offset field in the access address and returned to the processor. If they are not equal, it means that the data to be accessed has not been cached in the L1 cache (that is, "missing"). In the case of a cache miss, the L1 cache controller allocates a cache block and its corresponding tag for the data to be accessed, loads the data from the main memory into the allocated cache line, and adds the tag corresponding to the access address. The field is stored in the tag corresponding to the cache block.

In the existing L1 cache, the tag storage location in the tag storage unit is fixedly bound to each cache block (i.e., data storage location) in the data storage unit. When the data storage space of the L1 cache changes (for example, the cache When the location of the block is moved or changed), the L1 cache will not be able to address the corresponding data based on the tag and index. Therefore, the traditional L1 cache cannot support dynamic changes in storage space size.

In addition, it can be seen from the above analysis that the number of bits occupied by the offset field in the memory access address of the L1 cache is determined by the size of the cache block, and the tag field corresponds to the global memory to be accessed by the memory access request. Part of the address is also preset; only the index field The number of bits occupied will change with the size of the L1 cache, and changes in the index field will not affect external components such as global memory, only the internal addressing of the L1 cache. For example, for a cache block size of 128B and an 8-way set associative cache (i.e., each group contains 8 cache blocks), the offset determined by the cache block size occupies the lowest 7 bits of the access address (i.e., [ 6:0] part), when the cache size is 32KB (i.e. 128*8*32), there are 32 groups in total, and the index part of the access address occupies 5 bits, which is the [11:7] part; and if the cache The size becomes 64KB (128*8*64), then there are 64 groups in total, and the index part of the memory access address occupies 6 bits, that is, the [12:7] part. That is to say, when the size of the data storage space of the L1 cache changes, the size of the index field in the access address must be changed accordingly in order to perform correct addressing.

FIG. 1 shows a schematic structural module diagram of a cache memory 100 according to an embodiment of the present application. The cache memory 100 includes a controller 101, a tag storage unit 102 for saving tags, a data storage unit 103 composed of a plurality of cache blocks, and a mapping unit 104. Different from the fixed binding of tag storage locations and cache blocks in existing cache memories, there is a dynamic mapping between tag storage locations and data storage locations (ie, cache blocks) in the cache memory 100 . That is, each tag storage location is no longer fixedly bound to a cache block, but can be dynamically mapped to or bound to any cache block. In this embodiment, the mapping relationship between the tag storage location and the cache block is saved in the mapping unit 104 . The mapping unit 104 may, for example, store a one-to-one mapping relationship between tag serial numbers and cache block serial numbers in the form of a table. The tag serial number is used to indicate the location of each tag stored in the tag storage unit 102 . The cache block serial number is used to indicate the location of each cache block in the data storage unit 103.

Figure 2 shows a schematic diagram of the mapping relationship between tag storage locations and cache blocks according to an example of this application. As shown in Figure 2, there are k+1 tag storage locations in the tag storage unit, and there are n+1 cache blocks in the data storage unit, where n and k are both natural numbers, and n is greater than or equal to k. For example, the 1st tag t0 is currently mapped to the 6th cache block d5, the 2nd tag t1 is currently mapped to the 9th cache block d8,..., the k+1th tag is currently mapped to the 24th cache block d23 . It can be seen that the mapping relationship saved by the mapping unit 104 actually embodies or reflects the mapping between the tags currently saved in each storage location in the tag storage unit 102 and the data blocks currently saved in the corresponding cache block in the data storage unit 103 relation. When the location and storage space size of the L1 cache data storage unit 103 change, by adjusting the mapping between the storage location of the tag established by the mapping unit 104 and the storage location of the corresponding data, the tag storage unit 102 can be Each tag is mapped to the data storage unit 103 at the new location. individual cache blocks. Through this dynamic mapping and storage method, the number of tags and cache blocks that can be retrieved can be changed or adjusted, thereby supporting dynamic changes in the storage space of the L1 cache. Moreover, the L1 cache can be allowed to be moved or allocated to any part of the shared storage space. As long as the starting position of the L1 cache in the shared storage space is given, the data corresponding to each tag storage location can be relocated by adjusting the mapping unit 104. storage location.

In some embodiments, the number of cache blocks in the data storage unit 103 may be greater than the number of tags contained in the tag storage unit 102 . Each cache block can be bound to a tag or not. Each cache block has the tag binding bit set. The tag binding bit is used to indicate whether the cache block is bound to a tag. For example, when the cache block is bound to a certain tag, its tag binding bit can be set to 1, y, or T, etc., and the cache block is no longer bound. When specifying any tag, its tag binding bit can be set to 0, n, F, etc. Each cache block is only allowed to release the data resources when it is not bound to any tag, that is, it can participate in data replacement and can be used to store new data. In some embodiments, each cache block may also be set with a status bit to indicate whether the operation for the cache block has been completed. For example, when the controller 101 determines that all read and write operations for the data currently stored in the cache block have been completed, the status bit of the cache block can be set to 1, y, or T, etc.; conversely, its status can be set to 0, n or F etc. Each cache block is only allowed to release its data resources when no tags are bound and all read operations have been completed, that is, it can participate in data replacement and can be used to store new data.

In this embodiment, dynamic mapping or dynamic binding of tags and data in cache blocks is achieved by introducing the mapping unit 104 . Data in tags and cache blocks do not need to be updated synchronously. For example, when a tag stored in a certain storage location in the tag storage unit 102 is replaced with a new tag, a new cache block can be allocated in the data storage unit 103 for the data corresponding to the new tag, and the data corresponding to the new tag can be established in the mapping unit 104. The mapping between the new tag and the newly allocated cache block is sufficient, and the data in the cache block corresponding to the old tag originally saved in the storage location is still retained in the data storage unit 103 . Correspondingly, when the storage space of the L1 cache changes, for example, when the data storage unit 103 of the L1 cache is moved or allocated to another location in the shared storage space, as long as the starting location of the L1 cache in the shared storage space is given , the cache block can be reallocated from the starting position, as long as the data storage location corresponding to each tag storage location is repositioned in the mapping unit 104 accordingly. However, it should be understood that when the storage space of the L1 cache is changed or adjusted according to the user's configuration, what is located or found based on the mapping relationship re-established by the mapping unit 104 is the data in the newly allocated cache block, and the data in the storage space is not The data saved in the L1 cache before adjustment will not be retained, and the space it occupies can be replaced by other data.

In some embodiments, the mapping unit 104 can be implemented using a random access memory such as SRAM or DRAM, and in other cases, a data structure such as an array or a linked list is used to store the one-to-one mapping between the tag serial number and the cache block serial number. relation. Taking an array as an example, the number of elements in the array is the number of tags that can be stored in the tag storage unit 102 . The first element in the array stores the sequence number of the cache block currently corresponding to the first tag in the tag storage unit 102, and so on. In some embodiments, the mapping unit 104 may be implemented in the form of a register. For example, the mapping unit 104 may be implemented as a set of registers. Each register corresponds to the storage location of each tag in the tag storage unit 102, and each register The value is the sequence number of the cache block corresponding to the tag at the corresponding position. Such a mapping unit implemented in the form of a register can further reduce the cost and area occupied by the storage of mapping relationships in the L1 cache, and improve the speed of parsing the mapping relationships between tags and cache blocks.

Continuing to refer to FIG. 1 , when the cache memory 100 receives a memory access request sent by the memory access control unit LSU, the controller 101 parses the memory access address contained in the received memory access request. Locate the corresponding group according to the index field in the memory access address, and then compare the label field in the memory access request with the labels contained in the located group. If a matching tag can be found, it is a cache hit, indicating that the data to be accessed by the memory access request has been cached in the cache memory. If no matching tag is found after comparing all tags, it means that the data to be accessed by the memory access request has not been cached in the cache memory. At this time, the controller 101 needs to obtain the data from the next level memory (such as L2 cache). or main memory, etc.), the data to be accessed by the memory access request is read into the cache memory 100 .

In the case of a cache hit, the controller 101 determines the data storage location corresponding to the hit tag (ie, a certain cache block in the data storage unit 103) according to the mapping relationship saved in the mapping unit 104, and determines the data storage location according to the memory access address. The offset field extracts the data to be accessed by the memory access request from the corresponding cache block and returns it to the memory access control unit LSU of the processor as a response to the memory access request.

In the case of a cache miss, the controller 101 assigns a tag to the memory access request, for example, uses the tag part of the memory access address contained in the memory access request as a newly allocated tag, and saves the newly allocated tag in the tag storage. In the unit 102, it is necessary to replace the original label stored in one of the storage locations of the label storage unit 102 with the newly allocated label, thereby achieving label updating. In fact, the tag storage unit 102 allocates a storage location for the tag of the memory access request. At the same time, the controller 101 also needs to allocate a cache block for the memory access request in the data storage unit 103 so that the data to be accessed by the memory access request can be stored from the next level memory. according to. In order to establish a corresponding relationship between the tag assigned to the memory access request and the cache block, the controller 101 also needs to update the mapping relationship between the tag and the cache block in the mapping unit 104, so that the tag storage unit 102 is assigned to the access request. A mapping is established between the tag of the memory request and the cache block allocated to the memory access request in the data storage unit 103. For example, according to the serial number of the storage location of the tag in the tag storage unit 102, the cache block serial number corresponding to the tag serial number is searched in the mapping unit 104, and the cache block corresponding to the cache block serial number found in the data storage unit 103 is bound to its tag. The location is set to indicate that it is not bound to the tag, and the found cache block sequence number is replaced with the sequence number in the data storage unit 103 of the cache block assigned to the memory access request. After the corresponding mapping is established in the mapping unit 104, the tag binding bit of the cache block allocated to the memory access request is set to indicate that it has been bound to the tag. In this way, the data containing the memory access request can be read from the next-level memory and stored in the cache block allocated for the memory access request.

In some embodiments, the number of cache blocks in the data storage unit 103 may be greater than the number of tags contained in the tag storage unit 102 . Each cache block can be bound to a tag or not. Each cache block has tag binding bits and status bits set. The tag binding bit is used to indicate whether the cache block is bound to a tag. For example, when the cache block is bound to a certain tag, its tag binding bit can be set to 1, y, or T, etc., and the cache block is no longer bound. When specifying any tag, its tag binding bit can be set to 0, n, F, etc. The status bit is used to indicate whether the operation for this cache block has been completed. For example, when the controller 101 determines that all read and write operations for the data currently stored in the cache block have been completed, the status bit of the cache block can be set to 1, y, or T, etc.; conversely, its status can be set to 0, n or F etc. Each cache block is only allowed to release its data resources when no tags are bound and all read operations have been completed, that is, it can participate in data replacement and can be used to store new data.

In this embodiment, dynamic mapping or dynamic binding of tags and data in cache blocks is achieved by introducing the mapping unit 104 . Data in tags and cache blocks do not need to be updated synchronously. For example, when a tag stored in a certain storage location in the tag storage unit 102 is replaced with a new tag, a new cache block can be allocated in the data storage unit 103 for the data corresponding to the new tag, and the data corresponding to the new tag can be established in the mapping unit 104. The mapping between the new tag and the newly allocated cache block is sufficient, and the data in the cache block corresponding to the old tag originally saved in the storage location is still retained in the data storage unit 103 .

In some embodiments, the mapping unit 104 can be implemented using a random access memory such as SRAM or DRAM, and in other cases, a data structure such as an array or a linked list is used to store the one-to-one mapping between the tag serial number and the cache block serial number. relation. Taking an array as an example, the elements in the array The quantity is the number of tags that can be stored in the tag storage unit 102 . The first element in the array stores the sequence number of the cache block currently corresponding to the first tag in the tag storage unit 102, and so on. In some embodiments, the mapping unit 104 may be implemented in the form of a register. For example, the mapping unit 104 may be implemented as a set of registers. Each register corresponds to the storage location of each tag in the tag storage unit 102, and each register The value is the sequence number of the cache block corresponding to the tag at the corresponding position. Such a mapping unit implemented in the form of a register can further reduce the cost and area occupied by the storage of mapping relationships in the L1 cache, and improve the speed of parsing the mapping relationships between tags and cache blocks.

Figure 3 shows a method for dynamically sharing memory space in a parallel processor according to an embodiment of the present application, which mainly includes the following steps:

In step S1), the memory access control unit of the processor updates the starting positions of the local memory and the cache memory in the memory of the processor respectively according to the received settings of the local memory size and the cache memory size. In this embodiment, when users write applications on GPGPU processors using programming languages such as CUDA and OPENCL, they can re-adjust the local memory in the processor according to the selected programming model based on local memory or global memory. The storage space size of the memory and L1 cache to better improve the processor's execution performance for applications. If the user currently chooses programming based on local memory, the storage space of the local memory in the processor can be appropriately expanded. On the contrary, if the user currently chooses programming based on global memory, the storage space of the L1 cache in the processor can be appropriately expanded. In order to improve the execution performance of the processor for application programs without increasing the chip area and hardware cost, in this embodiment, the storage space of the local memory and L1 cache comes from the same random access memory (RAM) in the processor. ), and the storage space occupied by local memory and L1 cache is not fixed, but can dynamically change with the configuration provided by the user. For example, when the user chooses programming based on local memory, the space proportion of the local memory in the memory can be expanded; and when the user chooses programming based on global memory, the space proportion of the L1 cache in the memory can be expanded. The user can provide its current settings for the local memory size and cache memory size to the memory access control unit of the processor by, for example, calling the configuration interface provided by the processor, providing a configuration file, selecting corresponding configuration options, sending configuration commands, etc. The memory access control unit of the processor can divide storage spaces of corresponding sizes from the processor's memory to serve as local memory and cache memory respectively based on the settings it receives for the local memory size and cache memory size. Reallocation of the shared memory. In fact, for memory shared by local memory and L1 cache, every time for local memory and cache The re-division of the storage space occupied by the processor can be achieved by determining the new starting location of the local memory and cache memory in the processor's memory.

In some embodiments, when the memory access control unit of the processor divides storage spaces of corresponding sizes from the memory of the processor into local memory and cache memory according to the settings it receives for the local memory size and cache memory size. When buffering memory, the storage space of the local memory is allocated first, and then the storage space of the L1 cache is allocated. And for local memory, a corresponding size of storage space is allocated from the preset address in the processor's memory (such as the starting address of the memory) every time, so that the local memory occupies the low address space part of the shared memory space. . In this way, the starting position of the local memory does not need to change each time it is updated, and the new starting position of the updated cache memory in the processor's memory can be determined only based on the newly set local memory space size. This not only simplifies the storage space management process, but also when the size of the local memory is updated, the data in the shared local memory part before and after the update will not be lost. As mentioned above, for the L1 cache, when the data storage space size is updated, because the number of cache blocks involved changes, the index field in the memory access address also changes, so the data saved in the L1 cache before is updated. will be released or cleared. The L1 cache after the update cannot locate data cached before the update. For the L1 cache after storage space adjustment, when accessing a specific storage location in the L1 cache according to the starting position of the L1 cache in RAM, it is equivalent to adding an offset after the original address. This offset is also assigned to The size of local memory space.

In step S2) the memory access control unit of the processor updates the setting of the size of the index field in the memory access address of the cache memory according to the received setting of the cache memory size. As mentioned above, the number of bits occupied by the offset field in the L1 cache memory access address is determined by the size of the cache block, and the tag field corresponds to part of the global memory address to be accessed by the memory access request. , is also preset; only the number of bits occupied by the index field will change with the size of the L1 cache, and changes in the index field only affect the internal addressing of the L1 cache. When the size of the cache block and the number of tags contained in each group remain unchanged, when the size of the data storage space of the L1 cache changes, the size of the index field in the access address is changed accordingly to ensure correct addressing. For example, for a cache block size of 128B and an 8-way group associative cache (that is, each group contains 8 cache blocks), when the cache size is 32KB (that is, 128*8*32), there are 32 groups in total. The index field in the address occupies 5 bits; and if the cache size becomes 64KB (128*8*64), there are 64 groups in total, and the index field in the access address occupies 6 bits. That is, given the cache block size and the number of tags contained within each group, the index The number of bits occupied by a segment is set according to the number of groups in the L1 cache, which can be obtained by dividing the cache size by the cache block size and the number of tags contained in each group. When the memory access control unit of the processor determines the size of the index field in the memory access address of the L1 cache, it also means that the new number of groups saved in the L1 cache is determined. In one example, the memory access control unit may determine the starting position of the cache memory in the memory of the processor determined in step S1), the setting of the cache memory size, the new group number determined in step S2), etc. The parameters are included in the notification message indicating the adjustment space and are sent to the cache to facilitate the cache to realign and configure its data storage space.

Continuing to refer to Figure 3, in step S3), when the cache memory receives a notification message indicating an adjustment space sent from the memory access control unit of the processor, it can start from the starting position of the cache memory in the memory of the processor , and sequentially determine the new data storage location corresponding to each group and each cache block contained therein in the memory of the processor. As mentioned above, each cache block in the data storage unit of the cache memory needs to set a corresponding tag storage location in order to be correctly addressed. Therefore, in step S3), a new mapping relationship needs to be established between the redetermined locations of each cache block and each tag storage location of the tag storage unit of the cache memory. That is to say, a new mapping is established between each tag storage location in the tag storage unit of the cache and each new data storage location in the data storage unit after storage space adjustment, so that each tag storage location can be located based on the newly established mapping relationship. The cache block corresponding to the tag.

Through the above steps, the user can dynamically adjust the storage space allocated to local memory or L1 cache in the processor's memory at any time according to the actual application. When the memory access control unit of the processor receives a memory access request for the local memory, the updated starting position of the local memory can be used to locate the data to be accessed by the memory access request. When the memory access control unit responds to receiving a memory access request for the global memory, it first maps the address in the memory access request to the memory access address of the cache memory, and sends it to the cache memory, where the memory access request The address takes the updated index field size. When the cache memory receives a memory access request from the memory access control unit, it can locate the data to be accessed by the memory access request based on the mapping between the newly established tag storage location and the new data storage location. In such a solution, the same random access memory (RAM) space in the processor can be time-shared by the local memory and L1 cache, thereby achieving the purpose of reducing chip area and hardware costs. Because the storage space in local memory or L1 cache usually accounts for 80% of the area cost, using a dynamic shared storage space solution can save about 40% of the area cost. Moreover, this solution allows users to re-adjust the local memory in the processor according to the selected programming model based on local memory or global memory when writing applications. The storage space size of the memory and L1 cache can better improve the processor's execution performance for applications.

In some embodiments, after adjusting the storage space of the L1 cache through the above steps, when the L1 cache receives a memory access request from the memory access control unit, its controller controls the memory access included in the received memory access request. Address is parsed. Extract the corresponding index field from the cache address based on the updated index field size, locate the corresponding group based on the extracted index field, and then compare the label field in the cache request with each label contained in the located group. . If a matching tag can be found, the cache hits, indicating that the data accessed by the memory access request has been cached in the L1 cache. If no matching tag is found after comparing all tags, it means that the data to be accessed by the memory access request has not been cached in the L1 cache. At this time, the L1 cache needs to be retrieved from the next-level memory (such as the L2 cache or the main memory). The data to be accessed by the memory access request is read into the L1 cache. In the case of a cache hit, determine which cache block the hit tag corresponds to based on the newly established mapping relationship between tags and cache blocks in step S3), and extract the cache block from the cache block based on the offset field in the access address. The data to be accessed by the memory access request is returned to the memory access control unit as a response to the memory access request. In the case of a cache miss, the L1 cache controller replaces the tag stored in one of the tag storage locations of the tag storage unit with the tag field in the memory access address of the memory access request, and creates a newly created tag according to step S3) The mapping relationship with the cache block determines which cache block the selected tag storage location corresponds to, and then obtains the data to be accessed by the memory access request from the next-level memory and saves it in the cache block.

In some embodiments where the cache block of the L1 cache is set with a tag binding bit, in the case of a cache miss, the controller of the L1 cache allocates a tag storage location for the memory access request in the tag storage unit to store the access request. The tag field in the memory access address of the memory request is selected, and one of the multiple cache blocks not bound to the tag is selected from the data storage unit to be allocated to the memory access request; then the allocated tag storage location is stored in the mapping unit The tag binding bit of the original corresponding cache block is set to indicate that it is not bound to the tag; then a mapping relationship is established between the tag storage location and the cache block assigned to the memory access request, and the tag assigned to the memory access request is The tag binding bit of the cache block is set to indicate that it is bound to the tag; and the data to be accessed by the memory access request is obtained from the next level memory and saved in the cache block allocated for the memory access request.

In some embodiments of the present application, a processor that supports dynamic shared memory space is also provided. Except for the memory access control unit, memory and cache memory, the other components are the same as those of existing processors. This will not be described again. In this embodiment, the cache is a cache as received above in connection with Figures 1 and 2, which includes a controller, The label storage unit of the tag, the data storage unit and the mapping unit composed of multiple cache blocks. A part of the storage space of the memory of the processor is used as local memory, and the other part is used as a data storage unit of the cache memory. The memory may be implemented in the form of random access memory (RAM) such as SRAM, DRAM. The memory access control unit is configured to: update the starting positions of the local memory and the cache memory in the processor's memory according to the received settings for the local memory size and cache memory size; and based on the received settings for the local memory size and cache memory size; The setting of the cache size updates the setting of the size of the index field in the cache access address. For specific details, please refer to the content introduced above in conjunction with steps S1) and S2). The controller of the cache memory is configured to: determine the new data storage location corresponding to each cache block in the memory based on the size of the cache memory provided by the memory access control unit and its starting position in the memory of the processor, And a mapping between each tag storage location in the tag storage unit and the new data storage location of each cache block is established in the mapping unit. For specific details, please refer to the content introduced above in conjunction with step S3).

It should be understood that, in addition to being implemented in pure computer-readable program code, the modules such as memory access control units and controllers in the processor and cache memory mentioned in this article and the method steps for their execution can be completely implemented by The corresponding functional modules, processes or steps are logically programmed so that these modules implement the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, the controller, memory access control unit, etc. implemented in this way can be regarded as a kind of hardware component, and the devices included therein for realizing various functions can also be regarded as the internal structure of the hardware component. Or even, the means for implementing various functions can be considered as structures within hardware components as well as software modules implementing the relevant processes or method steps.

References in this specification to "various embodiments," "some embodiments," "one embodiment," or "an embodiment," etc., refer to a particular feature, structure, or property described in connection with the embodiment, including In at least one embodiment. Thus, appearances of the phrases "in various embodiments," "in some embodiments," "in one embodiment," or "in an embodiment" etc. in various places throughout this specification are not necessarily referring to the same implementation. example. Furthermore, specific features, structures, or properties may be combined in any suitable manner in one or more embodiments. Thus, a particular feature, structure, or property shown or described in connection with one embodiment may be combined, in whole or in part, without limitation with features, structures, or properties of one or more other embodiments so long as the combination is not non-unlimited. Logical or not working.

In this specification, the terms “including”, “having” and similar meanings are intended to Coverage is not exclusive, for example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed. , or optionally other steps or units inherent to such processes, methods, products or devices. "One" or "one" does not exclude more than one. In addition, each element in the drawings of this application is for schematic illustration only and is not drawn to scale.

Although the present application has been described through the above embodiments, the present application is not limited to the embodiments described here, and also includes various changes and changes made without departing from the scope of the present application.

Claims (10)

A method for dynamically sharing memory space in parallel processors, including: The memory access control unit of the processor updates the starting positions of the local memory and the cache memory in the processor's memory respectively according to the received settings of the local memory size and cache memory size, where the storage space of the memory One part is used as local memory, while the other part is used as the data storage unit of cache memory; The memory access control unit of the processor updates the setting of the size of the index field in the memory access address of the cache memory according to the received setting of the cache memory size; The cache memory determines the new data storage location corresponding to each cache block in the memory according to the cache memory size provided by the memory access control unit and its starting position in the processor's memory, and stores it in each tag. Create a mapping between the location and the new data storage location for each cache block. The method according to claim 1, further comprising: according to the settings of the local memory size and the cache memory size received by the memory access control unit of the processor, starting from a preset address in the memory of the processor. Divide a storage space of a corresponding size as a local memory, and then allocate a storage space for the cache memory, where the starting position of the local memory is the preset address. The method of claim 1, wherein the cache memory is a set associative cache, and wherein the size of the index field in the access address of the cache memory is preset based on the cache memory size divided by the cache memory. The results are determined by the cache block size and the number of tags contained in each group. The method according to claim 1, further comprising: in response to receiving a memory access request for the local memory, the memory access control unit of the processor uses the updated starting position of the local memory to locate the memory access request to access. The data. The method according to any one of claims 1-4, further comprising: in response to receiving a memory access request for the global memory, the memory access control unit of the processor maps the address in the memory access request into an address of the cache memory. fetch the memory address and send it to the cache, wherein the memory access address adopts the updated index field size; In response to receiving the memory access request from the memory access control unit, the cache memory locates the data to be accessed by the memory access request according to the established mapping between the tag storage location and the new data storage location. The method according to claim 5, wherein the cache memory responds to receiving the memory access request from the memory access control unit to locate the access according to the established mapping between the tag storage location and the new data storage location. The data to be accessed by the storage request includes: When a cache hit occurs, the cache block corresponding to the hit tag is determined based on the established mapping between the tag storage location and the new data storage location, and the data to be accessed by the memory access request is extracted from the cache block as the cache block for the memory access request. response to the request; Perform the following actions on a cache miss: Allocate a tag storage location for the memory access request to store the tag field in the memory access address of the memory access request, and select one of the multiple cache blocks not bound to the tag from the data storage unit of the cache memory to allocate Give the memory access request; The tag binding bit of the cache block originally corresponding to the allocated tag storage location is set to indicate that it is not bound to the tag, and then a mapping relationship is established between the tag storage location and the cache block allocated to the memory access request, and The cache block assigned to the fetch request has its tag binding bit set to indicate that it is bound to a tag; and The data to be accessed by this memory access request is obtained from the next level memory and stored in the cache block allocated for this memory access request. A processor that supports dynamic shared memory space, which includes a memory access control unit, a memory, and a cache memory. The cache memory includes a controller, a tag storage unit for saving tags, and data composed of multiple cache blocks. Storage unit and mapping unit; wherein a part of the storage space of the memory is used as local memory, and the other part is used as a data storage unit of the cache memory, where: The memory access control unit is configured to: update the starting positions of the local memory and the cache memory in the memory of the processor according to the received settings of the local memory size and the cache memory size; and according to the received settings The setting of the cache size updates the setting of the size of the index field in the cache access address; The controller of the cache memory is configured to: determine the new data storage location corresponding to each cache block in the memory based on the size of the cache memory provided by the memory access control unit and its starting position in the memory of the processor. , and establish a mapping between each tag storage location in the tag storage unit and the new data storage location of each cache block in the mapping unit. The processor according to claim 7, wherein the memory access control unit is further configured to: according to the settings of the local memory size and the cache memory size it receives, from a preset address in the memory Start by dividing storage space of corresponding size as local memory. store, and then allocate storage space for the cache memory, where the starting position of the local memory is the preset address. 7. The processor of claim 7, wherein said memory is implemented in the form of random access memory. The processor of claim 7, wherein the mapping unit is implemented in the form of a register.