[go: up one dir, main page]

WO2025025678A1 - Data access method based on cloud storage service, and offload card, system and device - Google Patents

Data access method based on cloud storage service, and offload card, system and device Download PDF

Info

Publication number
WO2025025678A1
WO2025025678A1 PCT/CN2024/087588 CN2024087588W WO2025025678A1 WO 2025025678 A1 WO2025025678 A1 WO 2025025678A1 CN 2024087588 W CN2024087588 W CN 2024087588W WO 2025025678 A1 WO2025025678 A1 WO 2025025678A1
Authority
WO
WIPO (PCT)
Prior art keywords
data access
target
identification
storage area
thread
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/087588
Other languages
French (fr)
Chinese (zh)
Inventor
但庚元
万士宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Cloud Computing Technologies Co Ltd
Original Assignee
Huawei Cloud Computing Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202311687305.XA external-priority patent/CN119440868A/en
Application filed by Huawei Cloud Computing Technologies Co Ltd filed Critical Huawei Cloud Computing Technologies Co Ltd
Publication of WO2025025678A1 publication Critical patent/WO2025025678A1/en
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication

Definitions

  • the present application relates to the field of Internet technology, and in particular to a data access method, offloading card, system and device based on cloud storage service.
  • data can be exchanged between different storage nodes (or processes).
  • the service module in the process can send an access request to the network module in the process, and then the network module sends the access request to the network module in other processes, so that the service modules in other processes can receive the access request and perform operations related to the access request in the service modules in other processes.
  • the business module corresponds to the network module one by one, which results in more memory resources being consumed when data is interacted between different processes, thus affecting the effectiveness of data access.
  • the present application provides a data access method, offload card, system and device based on cloud storage service, which solves the technical problem of consuming a large amount of memory resources when data is interacted between processes.
  • a data access method based on a cloud storage service wherein the cloud storage service is run on at least two cloud storage servers located in at least one cloud data center in one of multiple regions, an unloading card is inserted in a first cloud storage server among the at least two cloud storage servers and a client is installed, and the method is executed by the unloading card.
  • the method may include: first, the unloading card receives a data access request sent by the client, the data access request includes an identifier of a source service thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object, the source service thread is any one of the at least one service thread included in the client, the data access request is used to indicate to perform a preset operation on the target object in the target storage area, the preset operation includes a combination of one or more of the following operations: a read operation or a write operation, and the second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers.
  • the unloading card determines the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, the identification connection relationship including multiple identification combinations and link connection information corresponding to each identification combination in the multiple identification combinations, wherein the identification combination includes a business thread identification and a storage area identification; finally, the unloading card sends the data access request to the second cloud storage server through the link between the source business thread and the target storage area.
  • the method described in the first aspect can be executed by a computing device, which can be a terminal, an apparatus including a terminal, or a chip in a terminal, or the computing device can also be a network device, an apparatus including a network device, or a chip in a network device.
  • a computing device can be a terminal, an apparatus including a terminal, or a chip in a terminal
  • the computing device can also be a network device, an apparatus including a network device, or a chip in a network device.
  • the following description will be given by taking the execution of a computing device as an example.
  • an unloading card can correspond to at least one business thread, and the client (specifically, the source business thread) can send the data access request to the unloading card after receiving the data access request. Since the data access request includes the identifier of the source business thread, the identifier of the target storage area in the second cloud storage server, and the identifier of the target object, the unloading card can determine the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread and the identifier of the target storage area.
  • the unloading card can send the data access request to the second cloud storage server through the link, so that the data in the target storage area can be accessed, that is, the process of performing a preset operation on the target object in the target storage area.
  • the unloading card since the unloading card only needs to store a link between each business thread in at least one business thread and each storage area in the second cloud storage server, the memory resources consumed (or occupied) are limited, which can improve the effectiveness of data access.
  • the above-mentioned unloading card determines the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread and the identifier of the target storage area, specifically including: querying the identifier connection relationship based on the identifier of the source business thread and the identifier of the target storage area, and determining a target identifier combination that matches the identifier of the source business thread and the identifier of the target storage area from the identifier connection relationship; determining the link connection information corresponding to the target identifier combination as the target link connection information; and determining the link represented by the target link connection information as the link between the source business thread and the target storage area.
  • the target link connection information is the link connection information corresponding to the target identification combination, and the target identification combination includes the identification of the source business thread and the identification of the target storage area
  • the target link connection information can accurately and effectively reflect the link connection relationship between the source business thread and the target storage area.
  • the unloading card determines the link represented by the target link connection information as the link between the source business thread and the target storage area, that is, the link between each business thread and each storage area can be accurately and effectively determined, thereby improving the accuracy of data access.
  • one link connection information is a QP pair.
  • one link connection information is a QP pair, that is, the link between each service thread and each storage area can be characterized based on the QP pair, so that the offload card can accurately and effectively determine the link between the source service thread and the target storage area based on the QP pair representing the source service thread and the target storage area. In this way, the effectiveness of data access can be further improved.
  • the above-mentioned offload card includes at least one cache queue
  • the data access request is a request included in a target cache queue
  • the target cache queue is a cache queue in the at least one cache queue corresponding to the source service thread.
  • one service thread corresponds to one cache queue.
  • Data access requests sent by different service threads are requests included in the cache queues corresponding to different service threads.
  • Data access requests sent by different service threads can be added to the corresponding cache queues, and the unloading card can obtain the corresponding service threads from the different cache queues.
  • Each data access request can be allocated and arranged in an orderly manner, thereby improving the orderliness of data access.
  • the data access method further includes: when it is detected that the source service thread is interrupted, performing a reconstruction operation on the source service thread.
  • the unloading card performs a reconstruction operation on the source service thread, which can ensure that the source service thread continues to be in a normal state, thereby further improving the effectiveness of data access.
  • the data access method also includes: the offload card receives an access success response sent by the second cloud storage server, the access success response is used to notify the offload card that the preset operation is successfully performed on the target object in the target storage area; the offload card sends the access success response to the client.
  • the client after receiving the access success response sent by the unloading card, the client (specifically, the source service thread) can determine that the unloading card has successfully performed the preset operation on the target object in the target storage area. Before and after the unloading card accesses the data in the target storage area, the client (or the source service thread) only sends and receives messages through the unloading card, without the need to perceive the connection relationship between each service thread and each storage area, and can quickly access the relevant service data, thereby reducing the delay generated during the data access process.
  • the client specifically, the source service thread
  • the above-mentioned unloading card includes at least one cache queue.
  • the data access method Before determining the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, the data access method also includes: adding the data access request to the target cache queue, which is the cache queue corresponding to the source business thread in the at least one cache queue.
  • the unload card after receiving the data access request sent by the client, can add the data access request to the target cache queue corresponding to the source business thread, thereby ensuring that the data access request sent by each business thread can be added to the corresponding cache queue, and each data access request can be effectively cached, thereby improving the effectiveness and orderliness of data access.
  • the above-mentioned determining the link between the source business thread and the target storage area from the identification connection relationship based on the identification of the source business thread and the identification of the target storage area specifically includes: reading the data access request from the target cache queue, and determining the link from the identification connection relationship based on the identification of the source business thread and the identification of the target storage area included in the read data access request.
  • each service thread can follow single-thread polling, specifically, a service thread always sends the data access request it obtains to the offload card (specifically, the cache queue corresponding to the service thread in the offload card). Then the offload card can read the data access request corresponding to each service thread from the cache queue corresponding to each service thread, thereby realizing the relevant data In this way, the offload card and at least one service thread can be combined to obtain an RTC model, which can reduce the end-to-end delay in the data access process.
  • the offload card specifically, the cache queue corresponding to the service thread in the offload card
  • an offload card based on a cloud storage service runs on at least two cloud storage servers located in at least one cloud data center in one of multiple regions.
  • the offload card is inserted in a first cloud storage server among the at least two cloud storage servers and a client is installed.
  • the offload card includes a receiving module, a determining module and a sending module.
  • the receiving module is used to receive a data access request sent by the client.
  • the data access request includes an identifier of a source business thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object.
  • the source business thread is any one of the at least one business thread included in the client.
  • the data access request is used to indicate that an operation is performed on the target object in the target storage area.
  • a preset operation includes one or more of the following operations in combination: a read operation or a write operation, the second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers; the determination module is used to determine the link between the source business thread and the target storage area from an identification connection relationship according to the identification of the source business thread and the identification of the target storage area, the identification connection relationship includes multiple identification combinations and link connection information corresponding to each identification combination in the multiple identification combinations, wherein the identification combination includes a business thread identification and a storage area identification; the sending module is used to send the data access request to the second cloud storage server through the link between the source business thread and the target storage area.
  • the determination module is specifically used to query the identification connection relationship based on the identification of the source business thread and the identification of the target storage area, and determine the target identification combination that matches the identification of the source business thread and the identification of the target storage area from the identification connection relationship; the determination module is also specifically used to determine the link connection information corresponding to the target identification combination as the target link connection information; the determination module is also specifically used to determine the link represented by the target link connection information as the link between the source business thread and the target storage area.
  • one link connection information is a QP pair.
  • the above-mentioned offload card further includes at least one cache queue
  • the data access request is a request included in a target cache queue
  • the target cache queue is a cache queue in the at least one cache queue corresponding to the source service thread.
  • the above-mentioned unloading card further includes a reconstruction module; the reconstruction module is used to perform a reconstruction operation on the source service thread when it is detected that the source service thread is interrupted.
  • the receiving module is also used to receive an access success response sent by the second cloud storage server, and the access success response is used to notify the unloading card that the preset operation on the target object in the target storage area has been successfully performed; the sending module is also used to send the access success response to the client.
  • the above-mentioned unloading card also includes an adding module; the adding module is used to add the data access request to a target cache queue, and the target cache queue is a cache queue corresponding to the source service thread in the at least one cache queue.
  • the above-mentioned unloading card also includes a reading module; the reading module is used to read the data access request from the target cache queue; the determination module is specifically used to determine the link from the identification connection relationship based on the identification of the source business thread included in the read data access request and the identification of the target storage area.
  • the technical effect of the unloading card described in the second aspect can refer to the technical effect of the method described in the first aspect, and will not be repeated here.
  • a data access system based on a cloud storage service wherein the cloud storage service runs on at least two cloud storage servers located in at least one cloud data center in one of multiple regions, and the data access system includes the at least two cloud storage servers, wherein an offload card is inserted in a first cloud storage server of the at least two cloud storage servers and a client is installed thereon; the client sends a data access request to the offload card, wherein the data access request includes an identifier of a source business thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object, wherein the source business thread is any one of the at least one business thread included in the client, and the data access request is used to indicate an offload request to the target object in the target storage area.
  • the method comprises the steps of: performing a preset operation, wherein the preset operation includes one or more of the following operations: a read operation or a write operation, wherein the second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers; the unloading card determines a link between the source business thread and the target storage area from an identifier connection relationship according to an identifier of the source business thread and an identifier of the target storage area, and sends the data access request to the second cloud storage server through the link between the source business thread and the target storage area, wherein the identifier connection relationship includes multiple identifier combinations and link connection information corresponding to each identifier combination in the multiple identifier combinations, wherein the identifier combination includes a business thread identifier and a storage area identifier.
  • the technical effect of the data access system described in the third aspect can refer to the technical effect of the method described in the first aspect. I will not go into details here.
  • a data access device in a fourth aspect, includes: a module for executing the method described in the first aspect, such as a transceiver module and a processing module.
  • a module for executing the method described in the first aspect such as a transceiver module and a processing module.
  • the transceiver module is used to execute the transceiver function of the data access device
  • the processing module is used to execute functions of the data access device other than the transceiver function.
  • the transceiver module may include a sending module and a receiving module, wherein the sending module is used to implement the sending function of the unloading card described in the second aspect, and the receiving module is used to implement the receiving function of the unloading card described in the second aspect.
  • the data access device described in the fourth aspect may further include a storage module, wherein the storage module stores a program or instruction.
  • the processing module executes the program or instruction
  • the data access device may execute the method described in the first aspect.
  • the data access device described in the fourth aspect can be a terminal or a network device, or a chip (system) or other parts or components that can be set in a terminal or a network device, or a device that includes a terminal or a network device, and this application does not limit this.
  • a data access device comprising: a processor, wherein the processor is configured to execute the method described in the first aspect.
  • the data access device described in the fifth aspect may further include a transceiver.
  • the transceiver may be a transceiver circuit or an interface circuit.
  • the transceiver may be used for the data access device described in the fifth aspect to communicate with other devices.
  • the data access device described in the fifth aspect may further include a memory.
  • the memory may be integrated with the processor or may be separately provided.
  • the memory may be used to store the computer program and/or data involved in the method described in the first aspect.
  • the communication device described in the fifth aspect may be the terminal or network device described in the first aspect, or a chip (system) or other parts or components that may be set in the terminal or network device, or a device that includes the terminal or network device.
  • a data access device comprises: a processor, the processor is coupled to a memory, and the processor is used to execute a computer program stored in the memory, so that the data access device executes the method described in the first aspect.
  • the data access device described in the sixth aspect may further include a transceiver.
  • the transceiver may be a transceiver circuit or an interface circuit.
  • the transceiver may be used for the data access device described in the sixth aspect to communicate with other devices.
  • the data access device described in the sixth aspect may be the terminal or network device described in the first aspect, or a chip (system) or other parts or components that may be set in the terminal or network device, or a device that includes the terminal or network device.
  • a data access device comprising: a processor and a memory; the memory is used to store a computer program, and when the processor executes the computer program, the data access device executes the method described in the first aspect.
  • the data access device described in the seventh aspect may further include a transceiver.
  • the transceiver may be a transceiver circuit or an interface circuit.
  • the transceiver may be used for the data access device described in the seventh aspect to communicate with other devices.
  • the data access device described in the seventh aspect may be the terminal or network device described in the first aspect, or a chip (system) or other parts or components that may be set in the terminal or network device, or a device that includes the terminal or network device.
  • the technical effects of the data access device described in the seventh aspect can refer to the technical effects of the method described in the first aspect, and will not be repeated here.
  • a computing device cluster comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method described in the first aspect.
  • a computer program product which includes instructions, and when the instructions are executed by a computing device cluster, The computing device cluster executes the method described in the first aspect.
  • a computer-readable storage medium comprising computer program instructions.
  • the computing device cluster executes the method described in the first aspect.
  • FIG1 is a schematic diagram showing a principle of message transmission between different processes provided by an embodiment of the present application.
  • FIG2 is a schematic diagram showing a principle of data access provided by an embodiment of the present application.
  • FIG3 shows another schematic diagram of the principle of data access provided by an embodiment of the present application.
  • FIG4 shows a schematic diagram of a network architecture of a data access system provided in an embodiment of the present application
  • FIG5 is a schematic diagram showing a principle of a data access method provided in an embodiment of the present application.
  • FIG6 is a schematic diagram showing a principle of sending and receiving data access requests based on a cache queue provided in an embodiment of the present application
  • FIG7 is a schematic diagram showing the principle of another data access method provided in an embodiment of the present application.
  • FIG8 shows another schematic diagram of the principle of data access provided by an embodiment of the present application.
  • FIG9 shows a schematic structural diagram of an unloading card provided in an embodiment of the present application.
  • FIG10 is a schematic diagram showing the structure of a computing device provided in an embodiment of the present application.
  • FIG11 is a schematic diagram showing the structure of a computing device cluster provided in an embodiment of the present application.
  • FIG. 12 shows a schematic diagram of a network architecture of a computing device cluster provided in an embodiment of the present application.
  • Remote direct memory access means remote direct address access.
  • the local node can directly access the memory of the remote node.
  • the so-called direct means that the remote memory can be read and written by bypassing the complex transmission control protocol (TCP)/Internet protocol (IP) and other network protocol stacks of traditional Ethernet just like accessing local memory.
  • TCP complex transmission control protocol
  • IP Internet protocol
  • the offload card can access the data in the second cloud storage server (specifically the target storage area) by means of RDMA.
  • Queue pair is a concept in RDMA, which refers to a group of send queues and receive queues.
  • a QP pair is a link connection information
  • a link connection information can represent a link between a service thread and a storage area.
  • a thread is the smallest unit that an operating system can schedule operations on, and is an execution unit with a smaller granularity than a process.
  • a process can contain multiple threads, and the multiple threads can share all resources of the process.
  • the unload card can determine the link between the source business thread and the target storage area from the identification connection relationship based on the identification of any business thread (i.e., the source business thread) included in the client and the identification of the target storage area in the second cloud storage server.
  • the run to complete (RTC) model is a full user-mode scheduling model.
  • a program is generally divided into several different logical functions, but these logical functions will run on the core of a central processing unit (CPU), specifically, a request is processed by one thread during its life cycle.
  • the offload card and at least one service thread can be combined to obtain an RTC model to reduce the end-to-end delay in the data access process.
  • each submodule can be written using a traditional framework, so that the business module under the basic programming framework can send and receive messages through the network thread.
  • the network communication function provided by the network thread at least two modules need to be registered in a process: a network module and a business module, and the entity of a module is a thread in the process.
  • modules can pass messages to each other, and processes can also pass messages to each other. When a message is passed between a process and other processes, it needs to be transferred through their respective network modules.
  • the general process of message transmission is shown in Figure 1.
  • the business module 101 in process A needs to send an access request to the business module 104 in process B
  • the business module 101 needs to first send the access request to the network module 102 in process A; then, the network module 102 can send the access request to the network module 103 in process B; finally, the network module 103 can send the access request to the business module 104, that is, the data interaction process between process A and process B can be realized.
  • the business module in the process can also send a control message to the network module in the process (i.e., network module 102), and the control message is used to control the network module to implement a certain function, such as network link establishment.
  • the number of business threads configured in the client may be different from the number of network threads.
  • N business threads and M network threads may be configured, where N is an integer greater than or equal to 1, and M is an integer greater than or equal to 1, and N ⁇ M.
  • the client includes 5 business threads (i.e., business thread 201, business thread 202, business thread 203, business thread 204, and business thread 205), 6 queues (i.e., queue 206, queue 207, queue 208, queue 209, queue 210, and queue 211), and 6 network threads (i.e., network thread 212, network thread 213, network thread 214, network thread 215, network thread 216, and network thread 217), and one network thread corresponds to one queue.
  • business threads i.e., business thread 201, business thread 202, business thread 203, business thread 204, and business thread 205
  • 6 queues i.e., queue 206, queue 207, queue 208, queue 209, queue 210, and queue 211
  • 6 network threads i.e., network thread 212, network thread 213, network thread 214, network thread 215, network thread 216, and network thread 217), and one network thread corresponds to one queue.
  • the business thread 201 determines the network thread (e.g., network thread 214) that receives the access request, and sends and adds the access request to the queue (i.e., queue 208) corresponding to the network thread 214. Afterwards, the network thread 214 can read the access request from the queue 208 and perform data access in the corresponding storage area on the server.
  • the requests sent by N business threads can be evenly dispersed among M network threads, and the M network threads can evenly share the network pressure.
  • the business thread and the network thread run on different CPU cores, and the business thread sending an access request to the network thread can be understood as an inter-core task regarding thread switching, and this inter-core task will generate a certain delay overhead, resulting in a higher access delay. This affects the data access process in delay-sensitive scenarios (or scenarios with low delay requirements).
  • the number of service threads configured in the client and the number of network threads may also be the same.
  • N service threads and N network modules may be configured, i.e., one service thread corresponds to one network module (service thread and network module are bound one to one).
  • the client includes 4 service threads (i.e., service thread 301, service thread 302, service thread 303, and service thread 304) and 4 network modules (i.e., network module 305, network module 306, network module 307, and network module 308), and one service thread corresponds to (or is bound to) one network module.
  • service thread 301 sends an access request to the corresponding network module (i.e., network module 305).
  • network module 305 can perform data access in the corresponding storage area in the server.
  • the business thread corresponds to the network module one by one, and the business thread and the network thread run on the same CPU core.
  • N business threads send access requests through their bound network modules, and there is no thread switching process, which reduces the latency when accessing data.
  • this method consumes a lot of memory resources, and the problem of memory resource occupation needs to be solved.
  • an unloading card can correspond to at least one business thread, and the client (specifically, the source business thread) can send the data access request to the unloading card after receiving the data access request. Since the data access request includes the identifier of the source business thread, the identifier of the target storage area in the second cloud storage server, and the identifier of the target object, the unloading card can determine the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread and the identifier of the target storage area.
  • the unloading card can send the data access request to the second cloud storage server through the link, so that the data in the target storage area can be accessed, that is, the process of performing a preset operation on the target object in the target storage area.
  • the unloading card since the unloading card only needs to store a link between each business thread in at least one business thread and each storage area in the second cloud storage server, the memory resources consumed (or occupied) are limited, which can improve the effectiveness of data access.
  • the data access method, offloading card, system and device based on cloud storage service provided by the embodiment of the present application are applied to data access scenarios (including but not limited to large-scale storage cluster scenarios, delay-sensitive scenarios and bandwidth-type business load scenarios).
  • the offloading card After the offloading card receives the data access request sent by the client, it can determine the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, and send the data access request to the second cloud storage server through the link.
  • the data access method, offloading card, system and device based on cloud storage service provided in the embodiment of the present application can be applied to the data access system.
  • the data access system includes a computing device 401 and a computing device 402, wherein a client 4011 is inserted into the computing device 401 and an offloading card 4012 is installed.
  • the connection between the above-mentioned devices can be a wireless connection.
  • solid lines are used in FIG4 to illustrate.
  • the client 4011 includes at least one service thread, and the computing device 402 includes one or more storage areas.
  • a storage area included in the computing device 402 can also be understood as a memory (or disk) in the computing device 402.
  • the computing device 402 can also include at least one storage pool, and the one or more storage areas can belong to the same storage pool or different storage pools.
  • the client 4011 may send a data access request to the offload card 4012.
  • the offload card 4012 determines the link between the source service thread and the target storage area from the identification connection relationship according to the identification of the source service thread and the identification of the target storage area, and sends the data access request to the computing device 402 through the link.
  • a server may be installed and run in the computing device 402 .
  • the computing device may be a server, which may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (content delivery network, CDN), as well as big data and artificial intelligence platforms.
  • a server which may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (content delivery network, CDN), as well as big data and artificial intelligence platforms.
  • cloud services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (content delivery network, CDN), as well as big data and artificial intelligence platforms.
  • the two computing devices shown in FIG. 4 may be deployed in at least one cloud data center, which may belong to one region among multiple regions, and the two computing devices are used to provide cloud storage services.
  • the computing device may also be a terminal, which may also be referred to as a user equipment (UE), an access terminal, a subscriber unit, a user station, a mobile station (MS), a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent or a user device.
  • UE user equipment
  • MS mobile station
  • remote station remote terminal
  • the terminal in the embodiments of the present application may be a mobile phone, a cellular phone, a smart phone, a tablet computer, a wireless data card, a personal digital assistant (PDA), a wireless modem, a handheld device (handset), a laptop computer, a machine type communication (MTC) terminal, a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, an intelligent
  • the terminal of the present application may also be a vehicle-mounted module, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit built into a vehicle as one or more components or units.
  • the terminal may also be other devices with terminal functions, for example, the terminal may also be a device that functions as a terminal in device-to-device (D2D) communication.
  • D2D device-to-device
  • the embodiments of the present application do not limit the device form of the terminal.
  • the device for realizing the function of the terminal can be a terminal; it can also be a device that can support the terminal to realize the function, such as a chip system.
  • the device can be installed in the terminal, or used in combination with the terminal.
  • the chip system can be composed of chips, or it can include chips and other discrete devices.
  • the embodiment of the present application provides a data access method based on a cloud storage service
  • the cloud storage service is run on at least two cloud storage servers of at least one cloud data center in one of multiple regions
  • an offload card is inserted in the first cloud storage server of the at least two cloud storage servers and a client is installed
  • the method is executed by the offload card.
  • the method may include S101-S103.
  • the client sends a data access request to the offload card.
  • the data access request includes the identification of the source business thread, the identification of the target storage area in the second cloud storage server, and the identification of the target object.
  • the source business thread is any one of the at least one business thread included in the client.
  • the data access request is used to indicate a preset operation to be performed on the target object in the target storage area.
  • the preset operation includes one or a combination of the following operations: a read operation or a write operation.
  • the second cloud storage server is a cloud storage server other than the first cloud storage server among the above-mentioned at least two cloud storage servers.
  • the first cloud storage server is any one of the at least two cloud storage servers.
  • the second cloud storage server may include at least one storage area, and the target storage area is any one of the at least one storage area.
  • the client sending the data access request to the unloading card can specifically be the source service thread sending the data access request to the unloading card.
  • the user can initiate a data access request on the client, and then the client sends the data access request to any one of the at least one service threads (i.e., the source service thread) based on the scattering algorithm.
  • the source service thread can receive the data access request and send the data access request to the unloading card.
  • the offload card determines a link between the source service thread and the target storage area from an identification connection relationship according to an identification of the source service thread and an identification of the target storage area.
  • the identification connection relationship includes a plurality of identification combinations and link connection information corresponding to each identification combination in the plurality of identification combinations, and the identification combination includes a service thread identification and a storage area identification.
  • a link can be characterized by a link connection information.
  • the unloading card can first determine the link connection information that can characterize the link between the source business thread and the target storage area, so as to determine the link between the source business thread and the target storage area. That is, the above-mentioned unloading card determines the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, which can specifically include steps A to C.
  • Step A the offload card queries the identifier connection relationship based on the identifier of the source service thread and the identifier of the target storage area, and determines the target identifier combination matching the identifier of the source service thread and the identifier of the target storage area from the identifier connection relationship.
  • the identification connection relationship is a mapping relationship pre-stored by the client during the initialization phase, and the link connection information corresponding to an identification combination is used to characterize the link between the business thread corresponding to the business thread identifier included in the identification combination and the storage area corresponding to the storage area identifier included in the identification combination.
  • one link connection information may be a QP pair.
  • the QP pair may be uniquely identified based on an identifier or a serial number (number, referred to as QPN) of the QP pair.
  • Step B The offload card determines the link connection information corresponding to the target identifier combination as the target link connection information.
  • target link connection information is used to characterize the link between the source service thread and the target storage area.
  • the identification connection relationship includes 4 identification combinations (including identification combination 1, identification combination 2, identification combination 3 and identification combination 4) and link connection information corresponding to each of the 4 identification combinations.
  • the link connection information corresponding to identification combination 1 is link connection information 1
  • the link connection information corresponding to identification combination 2 is link connection information 2
  • the link connection information corresponding to identification combination 3 is link connection information 3
  • the link connection information corresponding to identification combination 4 is link connection information 4.
  • Identification combination 1 includes business thread identification 1 and storage area identification 1
  • identification combination 2 includes business thread identification 1 and storage area identification 2
  • identification combination 3 includes business thread identification 2 and storage area identification 1
  • identification combination 4 includes business thread identification 2 and storage area identification 2.
  • the offload card determines that the target link connection information is link connection information 2.
  • Step C The offload card determines the link represented by the target link connection information as the link between the source service thread and the target storage area.
  • the target link connection information is the link connection information corresponding to the target identification combination, and the target identification combination includes the identification of the source business thread and the identification of the target storage area
  • the target link connection information can accurately and effectively reflect the link connection relationship between the source business thread and the target storage area.
  • the unloading card determines the link represented by the target link connection information as the link between the source business thread and the target storage area, that is, the link between each business thread and each storage area can be accurately and effectively determined, thereby improving the accuracy of data access.
  • the offload card sends a data access request to the second cloud storage server through the link between the source service thread and the target storage area.
  • the second cloud storage server can access the data in the target storage area, specifically, perform a preset operation on the target object in the target storage area.
  • an unloading card can correspond to at least one business thread, and the client (specifically, the source business thread) can send the data access request to the unloading card after receiving the data access request. Since the data access request includes the identifier of the source business thread, the identifier of the target storage area in the second cloud storage server, and the identifier of the target object, the unloading card can determine the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread and the identifier of the target storage area.
  • the unloading card can send the data access request to the second cloud storage server through the link, so that the data in the target storage area can be accessed, that is, the process of performing a preset operation on the target object in the target storage area.
  • the unloading card since the unloading card only needs to store a link between each business thread in at least one business thread and each storage area in the second cloud storage server, the memory resources consumed (or occupied) are limited, which can improve the effectiveness of data access.
  • the above-mentioned unloading card includes at least one cache queue, and for the above-mentioned at least one business thread, one business thread corresponds to one cache queue.
  • the unloading card can also add the above-mentioned data access request to the target cache queue.
  • the unloading card can read the data access request from the target cache queue, and determine the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area included in the read data access request.
  • the target cache queue is a cache queue corresponding to the source service thread in the at least one cache queue
  • the data access request is a request included in the target cache queue
  • the client includes three business threads (i.e., business thread 501, business thread 502, and business thread 503)
  • the offload card includes three cache queues (i.e., cache queue 504, cache queue 505, and cache queue 506) and four QP pairs (including QP pair 507, QP pair 508, QP pair 509, and QP pair 510), wherein one business thread corresponds to one cache queue, and one QP pair is used to characterize the link between one business thread and one storage area.
  • service thread 502 sends the data access request to the unloading card after receiving the data access request, and the unloading card adds the data access request to the cache queue (specifically cache queue 505) corresponding to the service thread 502 after receiving the data access request. Then the unloading card reads the data access request from the cache queue 505, and determines the link between the service thread 502 and the target storage area from the identification connection relationship according to the identification of the service thread 502 included in the read data access request and the identification of the target storage area (assuming it is the link represented by QP pair 509). Finally, the unloading card sends the data access request to the second cloud storage server through the link represented by the QP pair 509.
  • the unloading card after receiving the data access request sent by the client, can add the data access request to the target cache queue corresponding to the source business thread.
  • the unloading card can read the data access request from the target cache queue, and determine the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread included in the read data access request and the identifier of the target storage area.
  • the unloading card sends the data access request to the second cloud storage server through the link.
  • each business thread can follow single-threaded polling, specifically a business thread always sends the data access request it obtains to the unloading card (specifically the cache queue corresponding to the business thread in the unloading card). Then the unloading card can read the data access request corresponding to each business thread from the cache queue corresponding to each business thread, thereby realizing the access process of related data. In this way, the unloading card and at least one business thread can be combined to obtain an RTC model, which can reduce the end-to-end delay in the data access process.
  • the above-mentioned offload card further includes a network processing unit (NPU), which is used to The data access request is added to the target cache queue and the data access request is read from the target cache queue.
  • NPU network processing unit
  • the unloading card can perform real-time detection on each of the at least one service thread.
  • the unloading card can perform a reconstruction operation on the source service thread.
  • the reconstruction operation is an operation for re-establishing the link, which can also be understood as a message keep-alive mechanism.
  • the interruption of a business thread can also be understood as the disconnection of the business thread or the business thread is in an abnormal state.
  • the unload card performs a reconstruction operation on the source business thread, which can ensure that the source business thread continues to be in a normal state, thereby further improving the effectiveness of data access.
  • the unloading card may have a successful data access or a failed data access.
  • the second cloud storage server can return a corresponding message to the unloading card to prompt the unloading card whether the data access to the target storage area is successful.
  • the data access method provided in the embodiment of the present application can also include S104-S105.
  • the offloading card receives an access success response sent by the second cloud storage server.
  • the access success response is used to notify the offloading card that a preset operation is successfully performed on the target object in the target storage area.
  • the second cloud storage server can determine whether the unloading card can successfully perform the preset operation on the target object in the target storage area. In the case where the unloading card successfully performs the preset operation on the target object in the target storage area, the second cloud storage server can send the access success response to the unloading card.
  • the unloading card successfully performing a preset operation on the target object in the target storage area can also be understood as the unloading card successfully accessing the data in the target storage area.
  • the client after receiving the access success response sent by the unloading card, the client (specifically, the source service thread) can determine that the unloading card has successfully performed the preset operation on the target object in the target storage area. Before and after the unloading card accesses the data in the target storage area, the client (or the source service thread) only sends and receives messages through the unloading card, and does not need to perceive the connection relationship between each service thread and each storage area, so it can quickly access the relevant service data, reducing the delay generated during the data access process.
  • the client specifically, the source service thread
  • the second cloud storage server may also send an access failure response to the unloading card, and the access failure response is used to notify the unloading card that the preset operation cannot be performed on the target object in the target storage area.
  • the unloading card may also send the access failure response to the client (specifically, the source service thread).
  • the above-mentioned unloading card may include a network data module and a network routing module.
  • the network data module is used to receive a data access request sent by the client, and receive an access success response sent by the second cloud storage server.
  • the network routing module is used to determine the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area.
  • an embodiment of the present application provides a data access system, which includes a first cloud storage server and a second cloud storage server, wherein the first cloud storage server includes a client 60 and an uninstall card 70, the client 60 includes a service thread 601, a service thread 602, a service thread 603, a service thread 604, and a service thread 605, and the second cloud storage server includes a storage pool 801 and a storage pool 802.
  • the storage pool 801 includes a storage area 8011, a storage area 8012, and a storage area 8013.
  • the storage pool 802 includes a storage area 8021, a storage area 8022, and a storage area 8023.
  • the data access process is specifically as follows: First, after receiving the data access request, the business thread 601 in the client 60 sends a data access request to the unloading card 70, and the data access request includes the identifier of the business thread 601 and the identifier of the storage area 8011. Afterwards, the unloading card 70 determines the link between the business thread 601 and the storage area 8011 from the identifier connection relationship according to the identifier of the business thread 601 and the identifier of the storage area 8011, and sends the data access request to the second cloud storage server through the link.
  • the unloading card 70 receives the access success response sent by the second cloud storage server, and the access success response is used to notify the unloading card 70 that the preset operation on the target object in the storage area 8011 is successfully performed. Finally, the unloading card 70 sends the access success response to the business thread 601 in the client 60. response.
  • the present application also provides an offload card based on a cloud storage service, the cloud storage service running on at least two cloud storage servers located in at least one cloud data center in one of the multiple regions, the offload card being inserted in a first cloud storage server of the at least two cloud storage servers and a client being installed.
  • the offload card 90 comprises: a receiving module 901 , a determining module 902 and a sending module 903 .
  • the receiving module 901 is used to receive a data access request sent by the client, wherein the data access request includes an identifier of a source business thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object.
  • the source business thread is any one of at least one business thread included in the client.
  • the data access request is used to indicate that a preset operation is performed on the target object in the target storage area.
  • the preset operation includes one or a combination of the following operations: a read operation or a write operation.
  • the second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers.
  • Determination module 902 is used to determine the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, wherein the identification connection relationship includes multiple identification combinations and link connection information corresponding to each identification combination in the multiple identification combinations, wherein the identification combination includes a business thread identification and a storage area identification.
  • the sending module 903 is used to send the data access request to the second cloud storage server through the link between the source service thread and the target storage area.
  • the determination module 902 is specifically used to query the identifier connection relationship based on the identifier of the source business thread and the identifier of the target storage area, and determine the target identifier combination that matches the identifier of the source business thread and the identifier of the target storage area from the identifier connection relationship.
  • the determination module 902 is further specifically configured to determine the link connection information corresponding to the target identifier combination as the target link connection information.
  • the determination module 902 is further specifically configured to determine the link represented by the target link connection information as the link between the source service thread and the target storage area.
  • one link connection information is a QP pair.
  • the above-mentioned offload card 90 further includes at least one cache queue, the data access request is a request included in a target cache queue, and the target cache queue is a cache queue in the at least one cache queue corresponding to the source service thread.
  • the above-mentioned unloading card 90 also includes a reconstruction module 904 .
  • the reconstruction module 904 is used to perform a reconstruction operation on the source service thread when it is detected that the source service thread is interrupted.
  • the receiving module 901 is further used to receive an access success response sent by the second cloud storage server, where the access success response is used to notify the unloading card that the preset operation is successfully performed on the target object in the target storage area.
  • the sending module 903 is further configured to send the access success response to the client.
  • the above-mentioned unloading card 90 also includes an adding module 905 .
  • the adding module 905 is used to add the data access request to a target cache queue, where the target cache queue is a cache queue in the at least one cache queue that corresponds to the source service thread.
  • the above-mentioned unloading card also includes a reading module 906 .
  • the reading module 906 is used to read the data access request from the target cache queue.
  • the determination module 902 is specifically configured to determine the link according to the identifier of the source service thread and the identifier of the target storage area included in the read data access request.
  • the receiving module 901, the determining module 902, the sending module 903, the reconstruction module 904, the adding module 905 and the reading module 906 can all be implemented by software, or can be implemented by hardware.
  • the implementation of the determining module 902 is described below by taking the determining module 902 as an example.
  • the implementation of the receiving module 901, the sending module 903, the reconstruction module 904, the adding module 905 and the reading module 906 can refer to the implementation of the determining module 902.
  • the determination module 902 may include code running on a computing instance.
  • the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more.
  • the determination module may include code running on multiple hosts/virtual machines/containers. It should be noted that the multiple hosts/virtual machines/containers used to run the code may be distributed in the same region or in different regions. Distributed in different regions. Furthermore, multiple hosts/virtual machines/containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region can include multiple AZs.
  • AZ availability zone
  • VPC virtual private cloud
  • multiple hosts/virtual machines/containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs.
  • VPC virtual private cloud
  • a VPC is set up in a region.
  • a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.
  • the determination module 902 may include at least one computing device, such as a server, etc.
  • the determination module 902 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD).
  • the PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.
  • the multiple computing devices included in the determination module 902 may be distributed in the same region or in different regions.
  • the multiple computing devices included in the determination module 902 may be distributed in the same AZ or in different AZs.
  • the multiple computing devices included in the determination module 902 may be distributed in the same VPC or in multiple VPCs.
  • the multiple computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
  • the determination module 902 can be used to execute any step in the data access method
  • the receiving module 901 can be used to execute any step in the data access method
  • the sending module 903 can be used to execute any step in the data access method
  • the reconstruction module 904 can be used to execute any step in the data access method
  • the adding module 905 can be used to execute any step in the data access method
  • the reading module 906 can also be used to execute any step in the data access method.
  • the steps that the receiving module 901, the determination module 902, the sending module 903, the reconstruction module 904, the adding module 905 and the reading module 906 are responsible for implementing can be specified as needed, and the receiving module 901, the determination module 902, the sending module 903, the reconstruction module 904, the adding module 905 and the reading module 906 respectively implement different steps in the data access method to realize all the functions of the unloading card.
  • the present application also provides a computing device 1000.
  • the computing device 1000 includes: a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008.
  • the processor 1004, the memory 1006, and the communication interface 1008 communicate with each other through the bus 1002.
  • the computing device 1000 may be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1000.
  • the bus 1002 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc.
  • the bus may be divided into an address bus, a data bus, a control bus, etc.
  • FIG. 10 is represented by only one line, but does not mean that there is only one bus or one type of bus.
  • the bus 1004 may include a path for transmitting information between various components of the computing device 1000 (e.g., the memory 1006, the processor 1004, and the communication interface 1008).
  • Processor 1004 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP).
  • processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP).
  • CPU central processing unit
  • GPU graphics processing unit
  • MP microprocessor
  • DSP digital signal processor
  • the memory 1006 may include a volatile memory (volatile memory), such as a random access memory (RAM).
  • volatile memory volatile memory
  • RAM random access memory
  • the processor 1004 may also include a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
  • ROM read-only memory
  • HDD hard disk drive
  • SSD solid state drive
  • the memory 1006 stores executable program codes, and the processor 1004 executes the executable program codes to respectively implement the functions of the aforementioned receiving module, determining module, sending module, reconstruction module, adding module, and reading module, thereby implementing the data access method. That is, the memory 1006 stores instructions for executing the data access method.
  • the communication interface 1003 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1000 and other devices or a communication network.
  • a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1000 and other devices or a communication network.
  • the present application also provides a computing device cluster.
  • the computing device cluster includes at least one computing device.
  • the device may be a server, such as a central server, an edge server, or a local server in a local data center.
  • the computing device may also be a terminal device such as a desktop computer, a notebook computer, or a smart phone.
  • the computing device cluster includes at least one computing device 1000.
  • the memory 1006 in one or more computing devices 1000 in the computing device cluster may store the same instructions for executing the data access method.
  • the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the data access method.
  • the combination of one or more computing devices 1000 may jointly execute instructions for executing the data access method.
  • the memory 1006 in different computing devices 1000 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the data access device. That is, the instructions stored in the memory 1006 in different computing devices 1000 can implement the functions of one or more modules of the receiving module, the determining module, the sending module, the adding module, the reading module and the reconstruction module.
  • one or more computing devices in the computing device cluster can be connected via a network.
  • the network can be a wide area network or a local area network, etc.
  • Figure 12 shows a possible implementation. As shown in Figure 12, two computing devices 1000A and 1000B are connected via a network. Specifically, the network is connected via a communication interface in each computing device.
  • the memory 1006 in the computing device 1000A stores instructions for executing the functions of the determination module, the addition module, the reading module, and the reconstruction module. At the same time, the memory 1006 in the computing device 1000B stores instructions for executing the functions of the receiving module and the sending module.
  • connection mode between the computing device clusters shown in FIG12 may be considered that the data access method provided by the present application requires a large number of receiving processes and sending processes, so it is considered that the functions implemented by the receiving module and the sending module are handed over to the computing device 1000B for execution.
  • the functions of the computing device 1000A shown in FIG12 may also be completed by multiple computing devices 1000.
  • the functions of the computing device 1000B may also be completed by multiple computing devices 1000.
  • the embodiment of the present application also provides another computing device cluster.
  • the connection relationship between the computing devices in the computing device cluster can be similar to the connection mode of the computing device cluster described in Figures 11 and 12.
  • the difference is that the memory 1006 in one or more computing devices 1000 in the computing device cluster can store the same instructions for executing the data access method.
  • the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the data access method.
  • the combination of one or more computing devices 1000 may jointly execute instructions for executing the data access method.
  • the embodiment of the present application also provides a computer program product including instructions.
  • the computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium.
  • the at least one computing device executes the data access method.
  • the embodiment of the present application also provides a computer-readable storage medium.
  • the computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media.
  • the available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).
  • the computer-readable storage medium includes instructions that instruct the computing device to execute a data access method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present application relates to the technical field of the Internet. Disclosed are a data access method based on a cloud storage service, and an offload card, a system and a device, which can consume relatively few memory resources and improve the effectiveness of data access. The method comprises: first, an offload card receiving a data access request sent by a client, wherein the data access request comprises an identifier of a source service thread, an identifier of a target storage region in a second cloud storage server, and an identifier of a target object, and the data access request is used for instructing that a preset operation is executed for the target object in the target storage region; then, the offload card determining from an identifier connection relationship a link between the source service thread and the target storage region on the basis of the identifier of the source service thread and the identifier of the target storage region; and finally, the offload card sending the data access request to the second cloud storage server by means of the link between the source service thread and the target storage region.

Description

基于云存储服务的数据访问方法、卸载卡、系统及设备Data access method, offloading card, system and device based on cloud storage service

本申请要求于2023年08月01日提交国家知识产权局、申请号为202310960462.7、申请名称为“一种消息处理的方法、装置及其他设备”的中国专利申请的优先权,以及于2023年12月08日提交国家知识产权局、申请号为202311687305.X、申请名称为“基于云存储服务的数据访问方法、卸载卡、系统及设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on August 1, 2023, with application number 202310960462.7 and application name “A method, device and other equipment for message processing”, and the priority of the Chinese patent application filed with the State Intellectual Property Office on December 8, 2023, with application number 202311687305.X and application name “Data access method, offloading card, system and equipment based on cloud storage service”, all contents of which are incorporated by reference in this application.

技术领域Technical Field

本申请涉及互联网技术领域,尤其涉及一种基于云存储服务的数据访问方法、卸载卡、系统及设备。The present application relates to the field of Internet technology, and in particular to a data access method, offloading card, system and device based on cloud storage service.

背景技术Background Art

在传统的分布式存储系统中,不同的存储节点(或进程)之间可以进行数据交互。具体的,当某一个进程需要访问其他进程时,该进程中的业务模块可以向该进程中的网络模块发送访问请求,之后该网络模块向其他进程中的网络模块发送该访问请求,以使得其他进程中的业务模块可以接收到该访问请求,并且在其他进程中的业务模块中执行与该访问请求相关的操作。In a traditional distributed storage system, data can be exchanged between different storage nodes (or processes). Specifically, when a process needs to access other processes, the service module in the process can send an access request to the network module in the process, and then the network module sends the access request to the network module in other processes, so that the service modules in other processes can receive the access request and perform operations related to the access request in the service modules in other processes.

但是,上述方法中,业务模块与网络模块一一对应,如此导致不同进程之间进行数据交互时会消耗较多的内存资源,影响了数据访问的有效性。However, in the above method, the business module corresponds to the network module one by one, which results in more memory resources being consumed when data is interacted between different processes, thus affecting the effectiveness of data access.

发明内容Summary of the invention

本申请提供一种基于云存储服务的数据访问方法、卸载卡、系统及设备,解决了进程间进行数据交互时消耗的内存资源较多的技术问题。The present application provides a data access method, offload card, system and device based on cloud storage service, which solves the technical problem of consuming a large amount of memory resources when data is interacted between processes.

第一方面,提供一种基于云存储服务的数据访问方法,该云存储服务运行在位于多个区域中的一个区域的至少一个云数据中心的至少两个云存储服务器上,该至少两个云存储服务器中的第一云存储服务器上插置有卸载卡并且安装有客户端,该方法由该卸载卡执行。该方法可以包括:首先,卸载卡接收该客户端发送的数据访问请求,该数据访问请求包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,该源业务线程为所述客户端中包括的至少一个业务线程中的任一个,该数据访问请求用于指示在该目标存储区域中对该目标对象执行预设操作,该预设操作包括以下的一项或几项操作的组合:读操作或者写操作,该第二云存储服务器为该至少两个云存储服务器中除该第一云存储服务器以外的云储存服务器。然后该卸载卡根据该源业务线程的标识以及该目标存储区域的标识从标识连接关系中确定该源业务线程与该目标存储区域之间的链路,该标识连接关系包括多个标识组合以及该多个标识组合中每个标识组合对应的链路连接信息,其中,该标识组合包括业务线程标识和存储区域标识;最终,该卸载卡通过该源业务线程与该目标存储区域之间的链路向该第二云存储服务器发送该数据访问请求。In a first aspect, a data access method based on a cloud storage service is provided, wherein the cloud storage service is run on at least two cloud storage servers located in at least one cloud data center in one of multiple regions, an unloading card is inserted in a first cloud storage server among the at least two cloud storage servers and a client is installed, and the method is executed by the unloading card. The method may include: first, the unloading card receives a data access request sent by the client, the data access request includes an identifier of a source service thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object, the source service thread is any one of the at least one service thread included in the client, the data access request is used to indicate to perform a preset operation on the target object in the target storage area, the preset operation includes a combination of one or more of the following operations: a read operation or a write operation, and the second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers. The unloading card then determines the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, the identification connection relationship including multiple identification combinations and link connection information corresponding to each identification combination in the multiple identification combinations, wherein the identification combination includes a business thread identification and a storage area identification; finally, the unloading card sends the data access request to the second cloud storage server through the link between the source business thread and the target storage area.

可以理解,第一方面所述的方法可以由计算设备执行,该计算设备可以是终端,包含终端的装置,或者终端内的芯片,或者该计算设备也可以是网络设备,包含网络设备的装置,或者网络设备内的芯片。方便理解,下文均以计算设备执行为例进行介绍。It can be understood that the method described in the first aspect can be executed by a computing device, which can be a terminal, an apparatus including a terminal, or a chip in a terminal, or the computing device can also be a network device, an apparatus including a network device, or a chip in a network device. For ease of understanding, the following description will be given by taking the execution of a computing device as an example.

本申请中,一个卸载卡可以对应至少一个业务线程,客户端(具体可以为源业务线程)在接收到数据访问请求之后可以向卸载卡发送该数据访问请求。由于该数据访问请求中包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,如此卸载卡可以根据该源业务线程的标识以及该目标存储区域的标识从标识连接关系中确定出该源业务线程与该目标存储区域之间的链路。又由于该链路为从源业务线程到目标存储区域进行数据访问的唯一链路,如此卸载卡可以通过该链路向第二云存储服务器发送该数据访问请求,从而能够访问目标存储区域中的数据,即为实现在目标存储区域中对目标对象执行预设操作的过程。本申请中,由于卸载卡中仅需要存储一份至少一个业务线程中每一个业务线程与第二云存储服务器中每一个存储区域之间的链路,消耗(或占用)的内存资源有限,能够提升数据访问的有效性。 In the present application, an unloading card can correspond to at least one business thread, and the client (specifically, the source business thread) can send the data access request to the unloading card after receiving the data access request. Since the data access request includes the identifier of the source business thread, the identifier of the target storage area in the second cloud storage server, and the identifier of the target object, the unloading card can determine the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread and the identifier of the target storage area. And since the link is the only link for data access from the source business thread to the target storage area, the unloading card can send the data access request to the second cloud storage server through the link, so that the data in the target storage area can be accessed, that is, the process of performing a preset operation on the target object in the target storage area. In the present application, since the unloading card only needs to store a link between each business thread in at least one business thread and each storage area in the second cloud storage server, the memory resources consumed (or occupied) are limited, which can improve the effectiveness of data access.

可选地,上述卸载卡根据所述源业务线程的标识以及所述目标存储区域的标识从标识连接关系中确定所述源业务线程与所述目标存储区域之间的链路具体包括:基于该源业务线程的标识以及该目标存储区域的标识查询标识连接关系,从该标识连接关系中确定出与该源业务线程的标识以及该目标存储区域的标识匹配的目标标识组合;将该目标标识组合对应的链路连接信息,确定为目标链路连接信息;将该目标链路连接信息表征的链路,确定为该源业务线程与该目标存储区域之间的链路。Optionally, the above-mentioned unloading card determines the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread and the identifier of the target storage area, specifically including: querying the identifier connection relationship based on the identifier of the source business thread and the identifier of the target storage area, and determining a target identifier combination that matches the identifier of the source business thread and the identifier of the target storage area from the identifier connection relationship; determining the link connection information corresponding to the target identifier combination as the target link connection information; and determining the link represented by the target link connection information as the link between the source business thread and the target storage area.

本申请中,由于目标链路连接信息为目标标识组合对应的链路连接信息,目标标识组合中包括源业务线程的标识以及目标存储区域的标识,因此目标链路连接信息能够准确、有效地反映出源业务线程与目标存储区域之间的链路连接关系。如此卸载卡将该目标链路连接信息表征的链路确定为源业务线程与目标存储区域之间的链路,即可以准确、有效地确定出每个业务线程与每个存储区域之间的链路,从而提升数据访问的准确性。In the present application, since the target link connection information is the link connection information corresponding to the target identification combination, and the target identification combination includes the identification of the source business thread and the identification of the target storage area, the target link connection information can accurately and effectively reflect the link connection relationship between the source business thread and the target storage area. In this way, the unloading card determines the link represented by the target link connection information as the link between the source business thread and the target storage area, that is, the link between each business thread and each storage area can be accurately and effectively determined, thereby improving the accuracy of data access.

可选地,一个链路连接信息为一个QP对。Optionally, one link connection information is a QP pair.

本申请中,一个链路连接信息为一个QP对,即可以基于QP对表征每个业务线程与每个存储区域之间的链路,从而卸载卡能够基于表征源业务线程与目标存储区域之间的QP对,准确、有效地确定出源业务线程与目标存储区域之间的链路。如此,可以进一步提升数据访问的有效性。In the present application, one link connection information is a QP pair, that is, the link between each service thread and each storage area can be characterized based on the QP pair, so that the offload card can accurately and effectively determine the link between the source service thread and the target storage area based on the QP pair representing the source service thread and the target storage area. In this way, the effectiveness of data access can be further improved.

可选地,上述卸载卡包括至少一个缓存队列,该数据访问请求为目标缓存队列中包括的请求,该目标缓存队列为该至少一个缓存队列中与该源业务线程对应的缓存队列。Optionally, the above-mentioned offload card includes at least one cache queue, the data access request is a request included in a target cache queue, and the target cache queue is a cache queue in the at least one cache queue corresponding to the source service thread.

本申请中,对于至少一个业务线程而言,一个业务线程对应一个缓存队列。不同业务线程发送的数据访问请求为不同业务线程对应的缓存队列中包括的请求。不同业务线程发送的数据访问请求可以被添加至相应的缓存队列,并且卸载卡可以从不同的缓存队列中获取到相应的业务线程。能够对每个数据访问请求进行有序分配、排列,提升了数据访问的有序性。In the present application, for at least one service thread, one service thread corresponds to one cache queue. Data access requests sent by different service threads are requests included in the cache queues corresponding to different service threads. Data access requests sent by different service threads can be added to the corresponding cache queues, and the unloading card can obtain the corresponding service threads from the different cache queues. Each data access request can be allocated and arranged in an orderly manner, thereby improving the orderliness of data access.

可选地,该数据访问方法还包括:当检测到上述源业务线程中断时,对该源业务线程执行重建操作。Optionally, the data access method further includes: when it is detected that the source service thread is interrupted, performing a reconstruction operation on the source service thread.

本申请中,当检测到源业务线程中断时,说明该源业务线程处于异常状态。此时卸载卡对该源业务线程执行重建操作,可以保证源业务线程持续处于正常状态,从而进一步提升数据访问的有效性。In this application, when the source service thread is detected to be interrupted, it indicates that the source service thread is in an abnormal state. At this time, the unloading card performs a reconstruction operation on the source service thread, which can ensure that the source service thread continues to be in a normal state, thereby further improving the effectiveness of data access.

可选地,该数据访问方法还包括:该卸载卡接收该第二云存储服务器发送的访问成功响应,该访问成功响应用于通知该卸载卡在该目标存储区域中成功对该目标对象进行该预设操作;该卸载卡向该客户端发送该访问成功响应。Optionally, the data access method also includes: the offload card receives an access success response sent by the second cloud storage server, the access success response is used to notify the offload card that the preset operation is successfully performed on the target object in the target storage area; the offload card sends the access success response to the client.

本申请中,客户端(具体可以为源业务线程)在接收到卸载卡发送的访问成功响应之后,即可以确定卸载卡在目标存储区域中成功对目标对象进行预设操作。在卸载卡访问目标存储区域中的数据前后,客户端(或源业务线程)仅通过卸载卡收发消息,无需感知每一个业务线程与每一个存储区域之间的连接关系,能够快速访问到相关业务数据,降低了数据访问过程中产生的时延。In the present application, after receiving the access success response sent by the unloading card, the client (specifically, the source service thread) can determine that the unloading card has successfully performed the preset operation on the target object in the target storage area. Before and after the unloading card accesses the data in the target storage area, the client (or the source service thread) only sends and receives messages through the unloading card, without the need to perceive the connection relationship between each service thread and each storage area, and can quickly access the relevant service data, thereby reducing the delay generated during the data access process.

可选地,上述卸载卡包括至少一个缓存队列,在上述根据该源业务线程的标识以及该目标存储区域的标识从标识连接关系中确定该源业务线程与该目标存储区域之间的链路之前,该数据访问方法还包括:将该数据访问请求添加至目标缓存队列,该目标缓存队列为该至少一个缓存队列中与该源业务线程对应的缓存队列。Optionally, the above-mentioned unloading card includes at least one cache queue. Before determining the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, the data access method also includes: adding the data access request to the target cache queue, which is the cache queue corresponding to the source business thread in the at least one cache queue.

本申请中,卸载卡在接收到客户端发送的数据访问请求之后,可以将该数据访问请求添加至与该源业务线程对应的目标缓存队列,能够保证每个业务线程发送的数据访问请求都能够被添加至相应的缓存队列,可以有效地缓存每一个数据访问请求,能够提升数据访问的有效性及有序性。In the present application, after receiving the data access request sent by the client, the unload card can add the data access request to the target cache queue corresponding to the source business thread, thereby ensuring that the data access request sent by each business thread can be added to the corresponding cache queue, and each data access request can be effectively cached, thereby improving the effectiveness and orderliness of data access.

可选地,上述根据该源业务线程的标识以及该目标存储区域的标识从标识连接关系中确定该源业务线程与该目标存储区域之间的链路具体包括:从该目标缓存队列中读取该数据访问请求,并且根据读取的该数据访问请求中包括的该源业务线程的标识以及该目标存储区域的标识从该标识连接关系中确定该链路。Optionally, the above-mentioned determining the link between the source business thread and the target storage area from the identification connection relationship based on the identification of the source business thread and the identification of the target storage area specifically includes: reading the data access request from the target cache queue, and determining the link from the identification connection relationship based on the identification of the source business thread and the identification of the target storage area included in the read data access request.

本申请中,每一个业务线程可以遵循单线程轮询,具体为一个业务线程始终将其获取到的数据访问请求发送至卸载卡(具体为卸载卡中与该业务线程对应的缓存队列)。然后卸载卡可以从每一个业务线程对应的缓存队列中读取该每一个业务线程对应的数据访问请求,从而实现相关数 据的访问过程。如此,卸载卡及至少一个业务线程可以组合得到RTC模型,能够降低数据访问过程中的端到端时延。In this application, each service thread can follow single-thread polling, specifically, a service thread always sends the data access request it obtains to the offload card (specifically, the cache queue corresponding to the service thread in the offload card). Then the offload card can read the data access request corresponding to each service thread from the cache queue corresponding to each service thread, thereby realizing the relevant data In this way, the offload card and at least one service thread can be combined to obtain an RTC model, which can reduce the end-to-end delay in the data access process.

第二方面,提供一种基于云存储服务的卸载卡,该云存储服务运行在位于多个区域中的一个区域的至少一个云数据中心的至少两个云存储服务器上,该至少两个云存储服务器中的第一云存储服务器上插置有该卸载卡并且安装有客户端,该卸载卡包括接收模块、确定模块以及发送模块;该接收模块,用于接收该客户端发送的数据访问请求,该数据访问请求包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,该源业务线程为该客户端中包括的至少一个业务线程中的任一个,该数据访问请求用于指示在该目标存储区域中对该目标对象执行预设操作,该预设操作包括以下的一项或几项操作的组合:读操作或者写操作,该第二云存储服务器为该至少两个云存储服务器中除该第一云存储服务器以外的云储存服务器;该确定模块,用于根据该源业务线程的标识以及该目标存储区域的标识从标识连接关系中确定该源业务线程与该目标存储区域之间的链路,该标识连接关系包括多个标识组合以及该多个标识组合中每个标识组合对应的链路连接信息,其中,该标识组合包括业务线程标识和存储区域标识;该发送模块,用于通过该源业务线程与该目标存储区域之间的链路向该第二云存储服务器发送该数据访问请求。In a second aspect, an offload card based on a cloud storage service is provided. The cloud storage service runs on at least two cloud storage servers located in at least one cloud data center in one of multiple regions. The offload card is inserted in a first cloud storage server among the at least two cloud storage servers and a client is installed. The offload card includes a receiving module, a determining module and a sending module. The receiving module is used to receive a data access request sent by the client. The data access request includes an identifier of a source business thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object. The source business thread is any one of the at least one business thread included in the client. The data access request is used to indicate that an operation is performed on the target object in the target storage area. A preset operation, the preset operation includes one or more of the following operations in combination: a read operation or a write operation, the second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers; the determination module is used to determine the link between the source business thread and the target storage area from an identification connection relationship according to the identification of the source business thread and the identification of the target storage area, the identification connection relationship includes multiple identification combinations and link connection information corresponding to each identification combination in the multiple identification combinations, wherein the identification combination includes a business thread identification and a storage area identification; the sending module is used to send the data access request to the second cloud storage server through the link between the source business thread and the target storage area.

可选地,该确定模块,具体用于基于该源业务线程的标识以及该目标存储区域的标识查询标识连接关系,从该标识连接关系中确定出与该源业务线程的标识以及该目标存储区域的标识匹配的目标标识组合;该确定模块,还具体用于将该目标标识组合对应的链路连接信息,确定为目标链路连接信息;该确定模块,还具体用于将该目标链路连接信息表征的链路,确定为该源业务线程与该目标存储区域之间的链路。Optionally, the determination module is specifically used to query the identification connection relationship based on the identification of the source business thread and the identification of the target storage area, and determine the target identification combination that matches the identification of the source business thread and the identification of the target storage area from the identification connection relationship; the determination module is also specifically used to determine the link connection information corresponding to the target identification combination as the target link connection information; the determination module is also specifically used to determine the link represented by the target link connection information as the link between the source business thread and the target storage area.

可选地,一个链路连接信息为一个QP对。Optionally, one link connection information is a QP pair.

可选地,上述卸载卡还包括至少一个缓存队列,该数据访问请求为目标缓存队列中包括的请求,该目标缓存队列为该至少一个缓存队列中与该源业务线程对应的缓存队列。Optionally, the above-mentioned offload card further includes at least one cache queue, the data access request is a request included in a target cache queue, and the target cache queue is a cache queue in the at least one cache queue corresponding to the source service thread.

可选地,上述卸载卡还包括重建模块;该重建模块,用于当检测到该源业务线程中断时,对该源业务线程执行重建操作。Optionally, the above-mentioned unloading card further includes a reconstruction module; the reconstruction module is used to perform a reconstruction operation on the source service thread when it is detected that the source service thread is interrupted.

可选地,该接收模块,还用于接收该第二云存储服务器发送的访问成功响应,该访问成功响应用于通知该卸载卡在该目标存储区域中成功对该目标对象进行该预设操作;该发送模块,还用于向该客户端发送该访问成功响应。Optionally, the receiving module is also used to receive an access success response sent by the second cloud storage server, and the access success response is used to notify the unloading card that the preset operation on the target object in the target storage area has been successfully performed; the sending module is also used to send the access success response to the client.

可选地,上述卸载卡还包括添加模块;该添加模块,用于将该数据访问请求添加至目标缓存队列,该目标缓存队列为该至少一个缓存队列中与该源业务线程对应的缓存队列。Optionally, the above-mentioned unloading card also includes an adding module; the adding module is used to add the data access request to a target cache queue, and the target cache queue is a cache queue corresponding to the source service thread in the at least one cache queue.

可选地,上述卸载卡还包括读取模块;该读取模块,用于从该目标缓存队列中读取该数据访问请求;该确定模块,具体用于根据读取的该数据访问请求中包括的该源业务线程的标识以及该目标存储区域的标识从该标识连接关系中确定该链路。Optionally, the above-mentioned unloading card also includes a reading module; the reading module is used to read the data access request from the target cache queue; the determination module is specifically used to determine the link from the identification connection relationship based on the identification of the source business thread included in the read data access request and the identification of the target storage area.

此外,第二方面所述的卸载卡的技术效果可以参考第一方面所述的方法的技术效果,此处不再赘述。In addition, the technical effect of the unloading card described in the second aspect can refer to the technical effect of the method described in the first aspect, and will not be repeated here.

第三方面,提供一种基于云存储服务的数据访问系统,该云存储服务运行在位于多个区域中的一个区域的至少一个云数据中心的至少两个云存储服务器上,该数据访问系统包括该至少两个云存储服务器,该至少两个云存储服务器中的第一云存储服务器上插置有卸载卡并且安装有客户端;该客户端向该卸载卡发送数据访问请求,该数据访问请求包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,该源业务线程为该客户端中包括的至少一个业务线程中的任一个,该数据访问请求用于指示在该目标存储区域中对该目标对象执行预设操作,该预设操作包括以下的一项或几项操作的组合:读操作或者写操作,该第二云存储服务器为该至少两个云存储服务器中除该第一云存储服务器以外的云储存服务器;该卸载卡根据该源业务线程的标识以及该目标存储区域的标识从标识连接关系中确定该源业务线程与该目标存储区域之间的链路,并且通过该源业务线程与该目标存储区域之间的链路向该第二云存储服务器发送该数据访问请求,该标识连接关系包括多个标识组合以及该多个标识组合中每个标识组合对应的链路连接信息,其中,该标识组合包括业务线程标识和存储区域标识。According to a third aspect, a data access system based on a cloud storage service is provided, wherein the cloud storage service runs on at least two cloud storage servers located in at least one cloud data center in one of multiple regions, and the data access system includes the at least two cloud storage servers, wherein an offload card is inserted in a first cloud storage server of the at least two cloud storage servers and a client is installed thereon; the client sends a data access request to the offload card, wherein the data access request includes an identifier of a source business thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object, wherein the source business thread is any one of the at least one business thread included in the client, and the data access request is used to indicate an offload request to the target object in the target storage area. The method comprises the steps of: performing a preset operation, wherein the preset operation includes one or more of the following operations: a read operation or a write operation, wherein the second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers; the unloading card determines a link between the source business thread and the target storage area from an identifier connection relationship according to an identifier of the source business thread and an identifier of the target storage area, and sends the data access request to the second cloud storage server through the link between the source business thread and the target storage area, wherein the identifier connection relationship includes multiple identifier combinations and link connection information corresponding to each identifier combination in the multiple identifier combinations, wherein the identifier combination includes a business thread identifier and a storage area identifier.

此外,第三方面所述的数据访问系统的技术效果可以参考第一方面所述的方法的技术效果, 此处不再赘述。In addition, the technical effect of the data access system described in the third aspect can refer to the technical effect of the method described in the first aspect. I will not go into details here.

第四方面,提供一种数据访问装置。该数据访问装置包括:用于执行第一方面所述的方法的模块,例如收发模块和处理模块。例如,收发模块,用于执行该数据访问装置的收发功能,处理模块,用于执行该数据访问装置除收发功能以外的功能。In a fourth aspect, a data access device is provided. The data access device includes: a module for executing the method described in the first aspect, such as a transceiver module and a processing module. For example, the transceiver module is used to execute the transceiver function of the data access device, and the processing module is used to execute functions of the data access device other than the transceiver function.

可选地,收发模块可以包括发送模块和接收模块。其中,发送模块用于实现第二方面所述的卸载卡的发送功能,接收模块用于实现第二方面所述的卸载卡的接收功能。Optionally, the transceiver module may include a sending module and a receiving module, wherein the sending module is used to implement the sending function of the unloading card described in the second aspect, and the receiving module is used to implement the receiving function of the unloading card described in the second aspect.

可选地,第四方面所述的数据访问装置还可以包括存储模块,该存储模块存储有程序或指令。当该处理模块执行该程序或指令时,使得该数据访问装置可以执行第一方面所述的方法。Optionally, the data access device described in the fourth aspect may further include a storage module, wherein the storage module stores a program or instruction. When the processing module executes the program or instruction, the data access device may execute the method described in the first aspect.

可以理解的是,第四方面所述的数据访问装置可以是终端或网络设备,也可以是可设置于终端或网络设备中的芯片(系统)或其他部件或组件,还可以是包含终端或网络设备的装置,本申请对此不做限定。It can be understood that the data access device described in the fourth aspect can be a terminal or a network device, or a chip (system) or other parts or components that can be set in a terminal or a network device, or a device that includes a terminal or a network device, and this application does not limit this.

此外,第四方面所述的数据访问装置的技术效果可以参考上述其他方面的技术效果,此处不再赘述。In addition, the technical effects of the data access device described in the fourth aspect can refer to the technical effects of the other aspects mentioned above, and will not be repeated here.

第五方面,提供一种数据访问装置。该数据访问装置包括:处理器,该处理器用于执行第一方面所述的方法。In a fifth aspect, a data access device is provided, wherein the data access device comprises: a processor, wherein the processor is configured to execute the method described in the first aspect.

在一种可能的设计方案中,第五方面所述的数据访问装置还可以包括收发器。该收发器可以为收发电路或接口电路。该收发器可以用于第五方面所述的数据访问装置与其他装置通信。In a possible design solution, the data access device described in the fifth aspect may further include a transceiver. The transceiver may be a transceiver circuit or an interface circuit. The transceiver may be used for the data access device described in the fifth aspect to communicate with other devices.

在一种可能的设计方案中,第五方面所述的数据访问装置还可以包括存储器。该存储器可以与处理器集成在一起,也可以分开设置。该存储器可以用于存储第一方面所述的方法所涉及的计算机程序和/或数据。In a possible design, the data access device described in the fifth aspect may further include a memory. The memory may be integrated with the processor or may be separately provided. The memory may be used to store the computer program and/or data involved in the method described in the first aspect.

在本申请实施例中,第五方面所述的通信装置可以为第一方面所述的终端或网络设备,或者可设置于该终端或网络设备中的芯片(系统)或其他部件或组件,或者包含该终端或网络设备的装置。In an embodiment of the present application, the communication device described in the fifth aspect may be the terminal or network device described in the first aspect, or a chip (system) or other parts or components that may be set in the terminal or network device, or a device that includes the terminal or network device.

此外,第五方面所述的数据访问装置的技术效果可以参考第一方面所述的方法的技术效果,此处不再赘述。In addition, the technical effects of the data access device described in the fifth aspect can refer to the technical effects of the method described in the first aspect, and will not be repeated here.

第六方面,提供一种数据访问装置。该通信装置包括:处理器,该处理器与存储器耦合,该处理器用于执行存储器中存储的计算机程序,以使得该数据访问装置执行第一方面所述的方法。In a sixth aspect, a data access device is provided. The communication device comprises: a processor, the processor is coupled to a memory, and the processor is used to execute a computer program stored in the memory, so that the data access device executes the method described in the first aspect.

在一种可能的设计方案中,第六方面所述的数据访问装置还可以包括收发器。该收发器可以为收发电路或接口电路。该收发器可以用于第六方面所述的数据访问装置与其他装置通信。In a possible design solution, the data access device described in the sixth aspect may further include a transceiver. The transceiver may be a transceiver circuit or an interface circuit. The transceiver may be used for the data access device described in the sixth aspect to communicate with other devices.

在本申请实施例中,第六方面所述的数据访问装置可以为第一方面所述的终端或网络设备,或者可设置于该终端或网络设备中的芯片(系统)或其他部件或组件,或者包含该终端或网络设备的装置。In an embodiment of the present application, the data access device described in the sixth aspect may be the terminal or network device described in the first aspect, or a chip (system) or other parts or components that may be set in the terminal or network device, or a device that includes the terminal or network device.

此外,第六方面所述的数据访问装置的技术效果可以参考第一方面所述的方法的技术效果,此处不再赘述。In addition, the technical effects of the data access device described in the sixth aspect can refer to the technical effects of the method described in the first aspect, and will not be repeated here.

第七方面,提供了一种数据访问装置,包括:处理器和存储器;该存储器用于存储计算机程序,当该处理器执行该计算机程序时,以使该数据访问装置执行第一方面所述的方法。In a seventh aspect, a data access device is provided, comprising: a processor and a memory; the memory is used to store a computer program, and when the processor executes the computer program, the data access device executes the method described in the first aspect.

在一种可能的设计方案中,第七方面所述的数据访问装置还可以包括收发器。该收发器可以为收发电路或接口电路。该收发器可以用于第七方面所述的数据访问装置与其他装置通信。In a possible design solution, the data access device described in the seventh aspect may further include a transceiver. The transceiver may be a transceiver circuit or an interface circuit. The transceiver may be used for the data access device described in the seventh aspect to communicate with other devices.

在本申请实施例中,第七方面所述的数据访问装置可以为第一方面所述的终端或网络设备,或者可设置于该终端或网络设备中的芯片(系统)或其他部件或组件,或者包含该终端或网络设备的装置。In an embodiment of the present application, the data access device described in the seventh aspect may be the terminal or network device described in the first aspect, or a chip (system) or other parts or components that may be set in the terminal or network device, or a device that includes the terminal or network device.

此外,第七方面所述的数据访问装置的技术效果可以参考第一方面所述的方法的技术效果,此处不再赘述。In addition, the technical effects of the data access device described in the seventh aspect can refer to the technical effects of the method described in the first aspect, and will not be repeated here.

第八方面,提供了一种计算设备集群,包括至少一个计算设备,每个计算设备包括处理器和存储器;该至少一个计算设备的处理器用于执行该至少一个计算设备的存储器中存储的指令,以使得该计算设备集群执行第一方面所述的方法。In an eighth aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method described in the first aspect.

第九方面,提供了一种包含指令的计算机程序产品,当该指令被计算设备集群运行时,使得 该计算设备集群执行第一方面所述的方法。In a ninth aspect, a computer program product is provided which includes instructions, and when the instructions are executed by a computing device cluster, The computing device cluster executes the method described in the first aspect.

第十方面,提供了一种计算机可读存储介质,包括计算机程序指令,当该计算机程序指令由计算设备集群执行时,该计算设备集群执行第一方面所述的方法。In a tenth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1示出了本申请实施例提供的一种不同进程之间进行消息传递的原理示意图;FIG1 is a schematic diagram showing a principle of message transmission between different processes provided by an embodiment of the present application;

图2示出了本申请实施例提供的一种进行数据访问的原理示意图;FIG2 is a schematic diagram showing a principle of data access provided by an embodiment of the present application;

图3示出了本申请实施例提供的另一种进行数据访问的原理示意图;FIG3 shows another schematic diagram of the principle of data access provided by an embodiment of the present application;

图4示出了本申请实施例提供的一种数据访问系统的网络架构示意图;FIG4 shows a schematic diagram of a network architecture of a data access system provided in an embodiment of the present application;

图5示出了本申请实施例提供的一种数据访问方法的原理示意图;FIG5 is a schematic diagram showing a principle of a data access method provided in an embodiment of the present application;

图6示出了本申请实施例提供的一种基于缓存队列收发数据访问请求的原理示意图;FIG6 is a schematic diagram showing a principle of sending and receiving data access requests based on a cache queue provided in an embodiment of the present application;

图7示出了本申请实施例提供的另一种数据访问方法的原理示意图;FIG7 is a schematic diagram showing the principle of another data access method provided in an embodiment of the present application;

图8示出了本申请实施例提供的另一种进行数据访问的原理示意图;FIG8 shows another schematic diagram of the principle of data access provided by an embodiment of the present application;

图9示出了本申请实施例提供的一种卸载卡的结构示意图;FIG9 shows a schematic structural diagram of an unloading card provided in an embodiment of the present application;

图10示出了本申请实施例提供的一种计算设备的结构示意图;FIG10 is a schematic diagram showing the structure of a computing device provided in an embodiment of the present application;

图11示出了本申请实施例提供的一种计算设备集群的结构示意图;FIG11 is a schematic diagram showing the structure of a computing device cluster provided in an embodiment of the present application;

图12示出了本申请实施例提供的一种计算设备集群的网络架构示意图。FIG. 12 shows a schematic diagram of a network architecture of a computing device cluster provided in an embodiment of the present application.

具体实施方式DETAILED DESCRIPTION

为便于理解本申请实施例提供的方案,在介绍本申请实施例提供的方案之前,对本申请涉及的一些概念做解释说明。To facilitate understanding of the solutions provided by the embodiments of the present application, some concepts involved in the present application are explained before introducing the solutions provided by the embodiments of the present application.

远程内存直接访问(remote direct memory access,RDMA),意为远程直接地址访问。通过RDMA,本端节点可以直接访问远端节点的内存。所谓直接,指的是可以像访问本地内存一样,绕过传统以太网复杂的传输控制协议(transmission control protocol,TCP)/互联网协议(internet protocol,IP)等网络协议栈读写远端内存,而这个过程远端是不感知的,并且整个读写过程中的大部分工作是由本端的硬件完成的。本申请实施例中,卸载卡可以通过RDMA的方式访问第二云存储服务器(具体为目标存储区域)中的数据。Remote direct memory access (RDMA) means remote direct address access. Through RDMA, the local node can directly access the memory of the remote node. The so-called direct means that the remote memory can be read and written by bypassing the complex transmission control protocol (TCP)/Internet protocol (IP) and other network protocol stacks of traditional Ethernet just like accessing local memory. The remote end is not aware of this process, and most of the work in the entire reading and writing process is completed by the local hardware. In the embodiment of the present application, the offload card can access the data in the second cloud storage server (specifically the target storage area) by means of RDMA.

队列对(queue pair,QP对)是RDMA中的概念,是指由一个发送队列(send queue)和一个接收队列(receive queue)组成的收发队列组。本申请实施例中,一个QP对为一个链路连接信息,一个链路连接信息可以表征一个业务线程与一个存储区域之间的链路。Queue pair (QP pair) is a concept in RDMA, which refers to a group of send queues and receive queues. In the embodiment of the present application, a QP pair is a link connection information, and a link connection information can represent a link between a service thread and a storage area.

线程,是操作系统能够进行运算调度的最小单位,是比进程粒度更小的执行单元。一个进程可以包含多个线程,该多个线程可以共享该进程的所有资源。本申请实施例中,卸载卡可以根据客户端中包括的任一业务线程(即源业务线程)的标识以及第二云存储服务器中目标存储区域的标识从标识连接关系中确定该源业务线程与该目标存储区域之间的链路。A thread is the smallest unit that an operating system can schedule operations on, and is an execution unit with a smaller granularity than a process. A process can contain multiple threads, and the multiple threads can share all resources of the process. In an embodiment of the present application, the unload card can determine the link between the source business thread and the target storage area from the identification connection relationship based on the identification of any business thread (i.e., the source business thread) included in the client and the identification of the target storage area in the second cloud storage server.

运行时完成(run to complete,RTC)模型,是一种全用户态调度模型。一个程序中一般会分为几个不同的逻辑功能,但这个几个逻辑功能会在一个中央处理器(central processing unit,CPU)的核上运行,具体为一个请求在生存周期内都由一个线程处理。本申请实施例中,卸载卡以及至少一个业务线程可以组合得到RTC模型,用于降低数据访问过程中的端到端时延。The run to complete (RTC) model is a full user-mode scheduling model. A program is generally divided into several different logical functions, but these logical functions will run on the core of a central processing unit (CPU), specifically, a request is processed by one thread during its life cycle. In the embodiment of the present application, the offload card and at least one service thread can be combined to obtain an RTC model to reduce the end-to-end delay in the data access process.

在传统的分布式存储系统中,各子模块可以采用传统框架进行编写,如此能够允许基础编程框架下的业务模块通过网络线程发送和接收消息。为了使用网络线程提供的网络通信功能,一个进程中至少需要注册两个模块:一个网络模块和一个业务模块,一个模块的实体是进程中的某一个线程。具体的,模块与模块之间可以相互传递消息,进程与进程之间也可以相互传递消息。在某一进程与其他进程之间传递消息时,需要通过各自的网络模块进行中转。传递消息的一般过程如图1所示,如果进程A中的业务模块101需要向进程B中的业务模块104发送访问请求,则业务模块101需要先向进程A中的网络模块102发送该访问请求;之后,网络模块102可以向进程B中的网络模块103发送该访问请求;最终,网络模块103可以向业务模块104发送该访问请求,即可以实现进程A与进程B之间的数据交互过程。 In a traditional distributed storage system, each submodule can be written using a traditional framework, so that the business module under the basic programming framework can send and receive messages through the network thread. In order to use the network communication function provided by the network thread, at least two modules need to be registered in a process: a network module and a business module, and the entity of a module is a thread in the process. Specifically, modules can pass messages to each other, and processes can also pass messages to each other. When a message is passed between a process and other processes, it needs to be transferred through their respective network modules. The general process of message transmission is shown in Figure 1. If the business module 101 in process A needs to send an access request to the business module 104 in process B, the business module 101 needs to first send the access request to the network module 102 in process A; then, the network module 102 can send the access request to the network module 103 in process B; finally, the network module 103 can send the access request to the business module 104, that is, the data interaction process between process A and process B can be realized.

另外,如图1所示,对于某一个进程内部的模块而言,该进程中的业务模块(例如业务模块101)也可以向该进程中的网络模块(即网络模块102)发送控制消息,该控制消息用于控制该网络模块实现某项功能,例如网络建链等。In addition, as shown in Figure 1, for a module within a process, the business module in the process (such as business module 101) can also send a control message to the network module in the process (i.e., network module 102), and the control message is used to control the network module to implement a certain function, such as network link establishment.

在一种可能的实现方式中,客户端中配置的业务线程的数量与网络线程的数量可以不同。例如,可以配置N个业务线程和M个网络线程,N为大于或等于1的整数,M为大于或等于1的整数,N≠M。示例性的,如图2所示,客户端中包括5个业务线程(即业务线程201、业务线程202、业务线程203、业务线程204以及业务线程205)、6个队列(即队列206、队列207、队列208、队列209、队列210以及队列211)以及6个网络线程(即网络线程212、网络线程213、网络线程214、网络线程215、网络线程216以及网络线程217),一个网络线程对应一个队列。具体的,业务线程201在接收到访问请求之后确定出接收该访问请求的网络线程(例如网络线程214),并且将访问请求发送并添加至与该网络线程214对应的队列(即队列208)中。之后,网络线程214可以从该队列208中读取该访问请求,并且在服务端中相应的存储区域进行数据访问。在图2示出的数据访问流程中,N个业务线程发送的请求可以均匀地打散到M个网络线程中,该M个网络线程能够均匀分担网络压力。但是,业务线程与网络线程运行在不同的CPU核上,业务线程向网络线程发送访问请求可以理解为一种关于线程切换的核间任务,而这种核间任务会产生一定的时延开销,带来较高的访问时延。从而影响在时延敏感场景(或者时延需求较低的场景)下的数据访问过程。In a possible implementation, the number of business threads configured in the client may be different from the number of network threads. For example, N business threads and M network threads may be configured, where N is an integer greater than or equal to 1, and M is an integer greater than or equal to 1, and N≠M. Exemplarily, as shown in FIG2 , the client includes 5 business threads (i.e., business thread 201, business thread 202, business thread 203, business thread 204, and business thread 205), 6 queues (i.e., queue 206, queue 207, queue 208, queue 209, queue 210, and queue 211), and 6 network threads (i.e., network thread 212, network thread 213, network thread 214, network thread 215, network thread 216, and network thread 217), and one network thread corresponds to one queue. Specifically, after receiving the access request, the business thread 201 determines the network thread (e.g., network thread 214) that receives the access request, and sends and adds the access request to the queue (i.e., queue 208) corresponding to the network thread 214. Afterwards, the network thread 214 can read the access request from the queue 208 and perform data access in the corresponding storage area on the server. In the data access process shown in FIG2, the requests sent by N business threads can be evenly dispersed among M network threads, and the M network threads can evenly share the network pressure. However, the business thread and the network thread run on different CPU cores, and the business thread sending an access request to the network thread can be understood as an inter-core task regarding thread switching, and this inter-core task will generate a certain delay overhead, resulting in a higher access delay. This affects the data access process in delay-sensitive scenarios (or scenarios with low delay requirements).

又一种可能的实现方式中,客户端中配置的业务线程的数量与网络线程的数量也可以相同。例如,可以配置N个业务线程和N个网络模块,即一个业务线程对应一个网络模块(业务线程与网络模块一对一绑定)。示例性的,如图3所示,客户端中包括4个业务线程(即业务线程301、业务线程302、业务线程303以及业务线程304)以及4个网络模块(即网络模块305、网络模块306、网络模块307以及网络模块308),一个业务线程对应(或绑定)一个网络模块。具体的,业务线程301在接收到访问请求之后,向对应的网络模块(即网络模块305)发送访问请求。网络模块305在接收到该访问请求之后可以在服务端中相应的存储区域进行数据访问。在图3示出的数据访问流程中,业务线程与网络模块一一对应,业务线程与网络线程运行在同一个CPU核上,N个业务线程通过自身绑定的网络模块将访问请求发送出去,不会存在线程切换的过程,在进行数据访问时的时延更低。但是,这种方式会消耗大量的内存资源,需要解决内存资源的占用问题。In another possible implementation, the number of service threads configured in the client and the number of network threads may also be the same. For example, N service threads and N network modules may be configured, i.e., one service thread corresponds to one network module (service thread and network module are bound one to one). Exemplary, as shown in Figure 3, the client includes 4 service threads (i.e., service thread 301, service thread 302, service thread 303, and service thread 304) and 4 network modules (i.e., network module 305, network module 306, network module 307, and network module 308), and one service thread corresponds to (or is bound to) one network module. Specifically, after receiving the access request, service thread 301 sends an access request to the corresponding network module (i.e., network module 305). After receiving the access request, network module 305 can perform data access in the corresponding storage area in the server. In the data access process shown in Figure 3, the business thread corresponds to the network module one by one, and the business thread and the network thread run on the same CPU core. N business threads send access requests through their bound network modules, and there is no thread switching process, which reduces the latency when accessing data. However, this method consumes a lot of memory resources, and the problem of memory resource occupation needs to be solved.

基于此,本申请实施例提供一种基于云存储服务的数据访问方法,一个卸载卡可以对应至少一个业务线程,客户端(具体可以为源业务线程)在接收到数据访问请求之后可以向卸载卡发送该数据访问请求。由于该数据访问请求中包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,如此卸载卡可以根据该源业务线程的标识以及该目标存储区域的标识从标识连接关系中确定出该源业务线程与该目标存储区域之间的链路。又由于该链路为从源业务线程到目标存储区域进行数据访问的唯一链路,如此卸载卡可以通过该链路向第二云存储服务器发送该数据访问请求,从而能够访问目标存储区域中的数据,即为实现在目标存储区域中对目标对象执行预设操作的过程。本申请实施例中,由于卸载卡中仅需要存储一份至少一个业务线程中每一个业务线程与第二云存储服务器中每一个存储区域之间的链路,消耗(或占用)的内存资源有限,能够提升数据访问的有效性。Based on this, the embodiment of the present application provides a data access method based on cloud storage service, an unloading card can correspond to at least one business thread, and the client (specifically, the source business thread) can send the data access request to the unloading card after receiving the data access request. Since the data access request includes the identifier of the source business thread, the identifier of the target storage area in the second cloud storage server, and the identifier of the target object, the unloading card can determine the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread and the identifier of the target storage area. Since the link is the only link for data access from the source business thread to the target storage area, the unloading card can send the data access request to the second cloud storage server through the link, so that the data in the target storage area can be accessed, that is, the process of performing a preset operation on the target object in the target storage area. In the embodiment of the present application, since the unloading card only needs to store a link between each business thread in at least one business thread and each storage area in the second cloud storage server, the memory resources consumed (or occupied) are limited, which can improve the effectiveness of data access.

本申请实施例提供的基于云存储服务的数据访问方法、卸载卡、系统及设备,应用于数据访问场景(包括但不限于大规模存储集群场景、时延敏感场景以及带宽型业务负载场景)。当卸载卡接收到客户端发送的数据访问请求之后,可以根据源业务线程的标识以及目标存储区域的标识从标识连接关系中确定该源业务线程与该目标存储区域之间的链路,并通过该链路向第二云存储服务器发送该数据访问请求。The data access method, offloading card, system and device based on cloud storage service provided by the embodiment of the present application are applied to data access scenarios (including but not limited to large-scale storage cluster scenarios, delay-sensitive scenarios and bandwidth-type business load scenarios). After the offloading card receives the data access request sent by the client, it can determine the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, and send the data access request to the second cloud storage server through the link.

为了使本领域普通人员更好地理解本申请的技术方案,下面将结合附图,对本申请实施例中的技术方案进行清楚、完整地描述。In order to enable ordinary persons in the art to better understand the technical solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

可以理解,本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实 施。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反,它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。It is understood that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

还应当理解的是,术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在,但并不排除一个或多个其他特征、整体、步骤、操作、元素和/或组件的存在或添加。It should also be understood that the term “comprising” indicates the presence of described features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements and/or components.

本申请实施例提供的基于云存储服务的数据访问方法、卸载卡、系统及设备可以应用于数据访问系统。如图4所示,该数据访问系统包括计算设备401和计算设备402,该计算设备401上插置有客户端4011并且安装有卸载卡4012。通常,在实际应用中上述各个设备之间的连接可以为无线连接,为了方便直观地表示各个设备之间的连接关系,图4中采用实线示意。The data access method, offloading card, system and device based on cloud storage service provided in the embodiment of the present application can be applied to the data access system. As shown in FIG4 , the data access system includes a computing device 401 and a computing device 402, wherein a client 4011 is inserted into the computing device 401 and an offloading card 4012 is installed. Generally, in practical applications, the connection between the above-mentioned devices can be a wireless connection. In order to conveniently and intuitively represent the connection relationship between the various devices, solid lines are used in FIG4 to illustrate.

其中,该客户端4011中包括至少一个业务线程,计算设备402中包括一个或多个存储区域。示例性的,计算设备402中包括的一个存储区域也可以理解为该计算设备402中的一个存储器(或磁盘)。计算设备402中还可以包括至少一个存储池,该一个或多个存储区域可以属于相同的存储池,也可以属于不同的存储池。The client 4011 includes at least one service thread, and the computing device 402 includes one or more storage areas. Exemplarily, a storage area included in the computing device 402 can also be understood as a memory (or disk) in the computing device 402. The computing device 402 can also include at least one storage pool, and the one or more storage areas can belong to the same storage pool or different storage pools.

具体的,客户端4011可以向卸载卡4012发送数据访问请求。之后该卸载卡4012根据源业务线程的标识以及目标存储区域的标识从标识连接关系中确定该源业务线程与该目标存储区域之间的链路,并通过该链路向计算设备402发送该数据访问请求。Specifically, the client 4011 may send a data access request to the offload card 4012. The offload card 4012 then determines the link between the source service thread and the target storage area from the identification connection relationship according to the identification of the source service thread and the identification of the target storage area, and sends the data access request to the computing device 402 through the link.

可选地,上述计算设备402中可以安装并运行有服务端。Optionally, a server may be installed and run in the computing device 402 .

一种示例中,上述计算设备可以是服务器,该服务器可以是独立的物理服务器,也可以是多个物理服务器构成的服务器集群或者分布式系统,还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、网络加速服务(content delivery network,CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。In one example, the computing device may be a server, which may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (content delivery network, CDN), as well as big data and artificial intelligence platforms.

应理解,图4中示出的两个计算设备可以部署在至少一个云数据中心中,该至少一个云数据中心可以属于多个区域中的一个区域,该两个计算设备用于提供云存储服务。It should be understood that the two computing devices shown in FIG. 4 may be deployed in at least one cloud data center, which may belong to one region among multiple regions, and the two computing devices are used to provide cloud storage services.

可以理解的是,图4中示出的两个计算设备仅为本申请实施例中的一种示例。本申请实施例对数据访问系统中包括的计算设备的数量不作具体限定。It is understandable that the two computing devices shown in Figure 4 are only an example in the embodiment of the present application. The embodiment of the present application does not specifically limit the number of computing devices included in the data access system.

另一种示例中,上述计算设备还可以是终端,终端也可以称为用户设备(user equipment,UE)、接入终端、用户单元(subscriber unit)、用户站、移动站(mobile station,MS)、移动台、远方站、远程终端、移动设备、用户终端、终端、无线通信设备、用户代理或用户装置。本申请的实施例中的终端可以是手机(mobile phone)、蜂窝电话(cellular phone)、智能电话(smart phone)、平板电脑(Pad)、无线数据卡、个人数字助理电脑(personal digital assistant,PDA)、无线调制解调器(modem)、手持设备(handset)、膝上型电脑(laptop computer)、机器类型通信(machine type communication,MTC)终端、带无线收发功能的电脑、虚拟现实(virtual reality,VR)终端、增强现实(augmented reality,AR)终端、智能家居设备(例如,冰箱、电视、空调、电表等)、智能机器人、机械臂、车间设备、工业控制(industrial control)中的无线终端、无人驾驶(self driving)中的无线终端、远程医疗(remote medical)中的无线终端、智能电网(smart grid)中的无线终端、运输安全(transportation safety)中的无线终端、智慧城市(smart city)中的无线终端、智慧家庭(smart home)中的无线终端、车载终端、具有终端功能的路边单元(road side unit,RSU)等、飞行设备(例如,智能机器人、热气球、无人机、飞机)等。本申请的终端还可以是作为一个或多个部件或者单元而内置于车辆的车载模块、车载模组、车载部件、车载芯片或者车载单元。终端还可以是其他具有终端功能的设备,例如,终端还可以是端到端(device to device,D2D)通信中担任终端功能的设备。In another example, the computing device may also be a terminal, which may also be referred to as a user equipment (UE), an access terminal, a subscriber unit, a user station, a mobile station (MS), a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent or a user device. The terminal in the embodiments of the present application may be a mobile phone, a cellular phone, a smart phone, a tablet computer, a wireless data card, a personal digital assistant (PDA), a wireless modem, a handheld device (handset), a laptop computer, a machine type communication (MTC) terminal, a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, an intelligent The terminal of the present application may also be a vehicle-mounted module, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit built into a vehicle as one or more components or units. The terminal may also be other devices with terminal functions, for example, the terminal may also be a device that functions as a terminal in device-to-device (D2D) communication.

本申请的实施例对终端的设备形态不做限定,用于实现终端的功能的装置可以是终端;也可以是能够支持终端实现该功能的装置,例如芯片系统。该装置可以被安装在终端中,或者和终端匹配使用。本申请实施例中,芯片系统可以由芯片构成,也可以包括芯片和其他分立器件。The embodiments of the present application do not limit the device form of the terminal. The device for realizing the function of the terminal can be a terminal; it can also be a device that can support the terminal to realize the function, such as a chip system. The device can be installed in the terminal, or used in combination with the terminal. In the embodiments of the present application, the chip system can be composed of chips, or it can include chips and other discrete devices.

结合上述实施例的描述,本申请实施例提供一种基于云存储服务的数据访问方法,该云存储服务运行在多个区域中的一个区域的至少一个云数据中心的至少两个云存储服务器上,该至少两个云存储服务器中的第一云存储服务器上插置有卸载卡并且安装有客户端,该方法由该卸载卡执行。具体的,如图5所示,该方法可以包括S101-S103。 In combination with the description of the above embodiments, the embodiment of the present application provides a data access method based on a cloud storage service, the cloud storage service is run on at least two cloud storage servers of at least one cloud data center in one of multiple regions, an offload card is inserted in the first cloud storage server of the at least two cloud storage servers and a client is installed, and the method is executed by the offload card. Specifically, as shown in FIG5 , the method may include S101-S103.

S101、客户端向卸载卡发送数据访问请求。S101. The client sends a data access request to the offload card.

其中,该数据访问请求包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,该源业务线程为该客户端中包括的至少一个业务线程中的任一个,该数据访问请求用于指示在该目标存储区域中对该目标对象进行预设操作,该预设操作包括以下的一项或几项操作的组合:读操作或者写操作,该第二云存储服务器为上述至少两个云存储服务器中除第一云存储服务器以外的云存储服务器。Among them, the data access request includes the identification of the source business thread, the identification of the target storage area in the second cloud storage server, and the identification of the target object. The source business thread is any one of the at least one business thread included in the client. The data access request is used to indicate a preset operation to be performed on the target object in the target storage area. The preset operation includes one or a combination of the following operations: a read operation or a write operation. The second cloud storage server is a cloud storage server other than the first cloud storage server among the above-mentioned at least two cloud storage servers.

应理解,第一云存储服务器为该至少两个云存储服务器中的任一个。第二云存储服务器中可以包括至少一个存储区域,该目标存储区域为该至少一个存储区域中的任一个。It should be understood that the first cloud storage server is any one of the at least two cloud storage servers. The second cloud storage server may include at least one storage area, and the target storage area is any one of the at least one storage area.

可以理解的是,客户端向卸载卡发送数据访问请求具体可以为上述源业务线程向卸载卡发送数据访问请求。具体的,用户可以在客户端上发起数据访问请求,之后客户端基于打散算法将该数据访问请求发送至该至少一个业务线程中的任一个业务线程(即源业务线程)。如此源业务线程可以接收到该数据访问请求,从而向卸载卡发送该数据访问请求。It is understandable that the client sending the data access request to the unloading card can specifically be the source service thread sending the data access request to the unloading card. Specifically, the user can initiate a data access request on the client, and then the client sends the data access request to any one of the at least one service threads (i.e., the source service thread) based on the scattering algorithm. In this way, the source service thread can receive the data access request and send the data access request to the unloading card.

S102、卸载卡根据源业务线程的标识以及目标存储区域的标识从标识连接关系中确定源业务线程与目标存储区域之间的链路。S102: The offload card determines a link between the source service thread and the target storage area from an identification connection relationship according to an identification of the source service thread and an identification of the target storage area.

其中,该标识连接关系包括多个标识组合以及该多个标识组合中每个标识组合对应的链路连接信息,该标识组合包括业务线程标识和存储区域标识。The identification connection relationship includes a plurality of identification combinations and link connection information corresponding to each identification combination in the plurality of identification combinations, and the identification combination includes a service thread identification and a storage area identification.

在本申请实施例的一种实现方式中,一条链路可以使用一个链路连接信息进行表征。卸载卡可以先确定出能够表征源业务线程与目标存储区域之间的链路的链路连接信息,从而可以确定出源业务线程与目标存储区域之间的链路。即上述卸载卡根据源业务线程的标识以及目标存储区域的标识从标识连接关系中确定源业务线程与目标存储区域之间的链路,具体可以包括步骤A-步骤C。In one implementation of the embodiment of the present application, a link can be characterized by a link connection information. The unloading card can first determine the link connection information that can characterize the link between the source business thread and the target storage area, so as to determine the link between the source business thread and the target storage area. That is, the above-mentioned unloading card determines the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, which can specifically include steps A to C.

步骤A、卸载卡基于源业务线程的标识以及目标存储区域的标识查询标识连接关系,从标识连接关系中确定出与源业务线程的标识以及目标存储区域的标识匹配的目标标识组合。Step A: the offload card queries the identifier connection relationship based on the identifier of the source service thread and the identifier of the target storage area, and determines the target identifier combination matching the identifier of the source service thread and the identifier of the target storage area from the identifier connection relationship.

应理解,该标识连接关系为客户端在初始化阶段预先存储的映射关系,一个标识组合对应的链路连接信息用于表征该标识组合包括的业务线程标识对应的业务线程与该标识组合包括的存储区域标识对应的存储区域之间的链路。It should be understood that the identification connection relationship is a mapping relationship pre-stored by the client during the initialization phase, and the link connection information corresponding to an identification combination is used to characterize the link between the business thread corresponding to the business thread identifier included in the identification combination and the storage area corresponding to the storage area identifier included in the identification combination.

可选地,一个链路连接信息可以为一个QP对。在具体的使用过程中,可以基于一个QP对的标识或系列号(number,简称为QPN)对该QP对进行唯一标识。Optionally, one link connection information may be a QP pair. In a specific use process, the QP pair may be uniquely identified based on an identifier or a serial number (number, referred to as QPN) of the QP pair.

步骤B、卸载卡将目标标识组合对应的链路连接信息,确定为目标链路连接信息。Step B: The offload card determines the link connection information corresponding to the target identifier combination as the target link connection information.

可以理解的是,该目标链路连接信息用于表征源业务线程与目标存储区域之间的链路。It can be understood that the target link connection information is used to characterize the link between the source service thread and the target storage area.

示例性的,以下表1为本申请实施例提供的标识连接关系的一种示例。具体的,该标识连接关系包括4个标识组合(包括标识组合1、标识组合2、标识组合3以及标识组合4)以及该4个标识组合各自对应的链路连接信息。具体的,标识组合1对应的链路连接信息为链路连接信息1,标识组合2对应的链路连接信息为链路连接信息2,标识组合3对应的链路连接信息为链路连接信息3,标识组合4对应的链路连接信息为链路连接信息4。标识组合1中包括业务线程标识1和存储区域标识1,标识组合2中包括业务线程标识1和存储区域标识2,标识组合3中包括业务线程标识2和存储区域标识1,标识组合4中包括业务线程标识2和存储区域标识2。Exemplarily, the following Table 1 is an example of an identification connection relationship provided in an embodiment of the present application. Specifically, the identification connection relationship includes 4 identification combinations (including identification combination 1, identification combination 2, identification combination 3 and identification combination 4) and link connection information corresponding to each of the 4 identification combinations. Specifically, the link connection information corresponding to identification combination 1 is link connection information 1, the link connection information corresponding to identification combination 2 is link connection information 2, the link connection information corresponding to identification combination 3 is link connection information 3, and the link connection information corresponding to identification combination 4 is link connection information 4. Identification combination 1 includes business thread identification 1 and storage area identification 1, identification combination 2 includes business thread identification 1 and storage area identification 2, identification combination 3 includes business thread identification 2 and storage area identification 1, and identification combination 4 includes business thread identification 2 and storage area identification 2.

表1
Table 1

假设源业务线程的标识为业务线程标识1,目标存储区域的标识为存储区域标识2,则卸载卡确定目标链路连接信息为链路连接信息2。Assuming that the identifier of the source service thread is service thread identifier 1 and the identifier of the target storage area is storage area identifier 2, the offload card determines that the target link connection information is link connection information 2.

步骤C、卸载卡将目标链路连接信息表征的链路,确定为源业务线程与目标存储区域之间的链路。 Step C: The offload card determines the link represented by the target link connection information as the link between the source service thread and the target storage area.

本申请实施例中,由于目标链路连接信息为目标标识组合对应的链路连接信息,目标标识组合中包括源业务线程的标识以及目标存储区域的标识,因此目标链路连接信息能够准确、有效地反映出源业务线程与目标存储区域之间的链路连接关系。如此卸载卡将该目标链路连接信息表征的链路确定为源业务线程与目标存储区域之间的链路,即可以准确、有效地确定出每个业务线程与每个存储区域之间的链路,从而提升数据访问的准确性。In the embodiment of the present application, since the target link connection information is the link connection information corresponding to the target identification combination, and the target identification combination includes the identification of the source business thread and the identification of the target storage area, the target link connection information can accurately and effectively reflect the link connection relationship between the source business thread and the target storage area. In this way, the unloading card determines the link represented by the target link connection information as the link between the source business thread and the target storage area, that is, the link between each business thread and each storage area can be accurately and effectively determined, thereby improving the accuracy of data access.

S103、卸载卡通过源业务线程与目标存储区域之间的链路向第二云存储服务器发送数据访问请求。S103: The offload card sends a data access request to the second cloud storage server through the link between the source service thread and the target storage area.

可以理解的是,第二云存储服务器在通过源业务线程与目标存储区域之间的链路接收到该数据访问请求之后,可以访问该目标存储区域中的数据,具体为在该目标存储区域中对目标对象执行预设操作。It is understandable that after receiving the data access request through the link between the source service thread and the target storage area, the second cloud storage server can access the data in the target storage area, specifically, perform a preset operation on the target object in the target storage area.

本申请实施例中,一个卸载卡可以对应至少一个业务线程,客户端(具体可以为源业务线程)在接收到数据访问请求之后可以向卸载卡发送该数据访问请求。由于该数据访问请求中包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,如此卸载卡可以根据该源业务线程的标识以及该目标存储区域的标识从标识连接关系中确定出该源业务线程与该目标存储区域之间的链路。又由于该链路为从源业务线程到目标存储区域进行数据访问的唯一链路,如此卸载卡可以通过该链路向第二云存储服务器发送该数据访问请求,从而能够访问目标存储区域中的数据,即为实现在目标存储区域中对目标对象执行预设操作的过程。本申请实施例中,由于卸载卡中仅需要存储一份至少一个业务线程中每一个业务线程与第二云存储服务器中每一个存储区域之间的链路,消耗(或占用)的内存资源有限,能够提升数据访问的有效性。In the embodiment of the present application, an unloading card can correspond to at least one business thread, and the client (specifically, the source business thread) can send the data access request to the unloading card after receiving the data access request. Since the data access request includes the identifier of the source business thread, the identifier of the target storage area in the second cloud storage server, and the identifier of the target object, the unloading card can determine the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread and the identifier of the target storage area. And since the link is the only link for data access from the source business thread to the target storage area, the unloading card can send the data access request to the second cloud storage server through the link, so that the data in the target storage area can be accessed, that is, the process of performing a preset operation on the target object in the target storage area. In the embodiment of the present application, since the unloading card only needs to store a link between each business thread in at least one business thread and each storage area in the second cloud storage server, the memory resources consumed (or occupied) are limited, which can improve the effectiveness of data access.

在本申请实施例的一种实现方式中,上述卸载卡包括至少一个缓存队列,对于上述至少一个业务线程而言,一个业务线程对应一个缓存队列。在上述卸载卡根据源业务线程的标识以及目标存储区域的标识从标识连接关系中确定源业务线程与目标存储区域之间的链路之前,该卸载卡还可以将上述数据访问请求添加至目标缓存队列。之后卸载卡可以从该目标缓存队列中读取该数据访问请求,并且根据读取的该数据访问请中包括的源业务线程的标识以及目标存储区域的标识从该标识连接关系中确定该源业务线程与该目标存储区域之间的链路。In one implementation of the embodiment of the present application, the above-mentioned unloading card includes at least one cache queue, and for the above-mentioned at least one business thread, one business thread corresponds to one cache queue. Before the above-mentioned unloading card determines the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, the unloading card can also add the above-mentioned data access request to the target cache queue. Afterwards, the unloading card can read the data access request from the target cache queue, and determine the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area included in the read data access request.

具体的,该目标缓存队列为上述至少一个缓存队列中与该源业务线程对应的缓存队列,该数据访问请求为该目标缓存队列中包括的请求。Specifically, the target cache queue is a cache queue corresponding to the source service thread in the at least one cache queue, and the data access request is a request included in the target cache queue.

示例性的,如图6所示,客户端中包括3个业务线程(即业务线程501、业务线程502以及业务线程503),卸载卡中包括3个缓存队列(即缓存队列504、缓存队列505以及缓存队列506)以及4个QP对(包括QP对507、QP对508、QP对509以及QP对510),其中,一个业务线程对应一个缓存队列,一个QP对用于表征一个业务线程与一个存储区域之间的链路。Exemplarily, as shown in FIG6 , the client includes three business threads (i.e., business thread 501, business thread 502, and business thread 503), and the offload card includes three cache queues (i.e., cache queue 504, cache queue 505, and cache queue 506) and four QP pairs (including QP pair 507, QP pair 508, QP pair 509, and QP pair 510), wherein one business thread corresponds to one cache queue, and one QP pair is used to characterize the link between one business thread and one storage area.

假设上述客户端中的源业务线程为业务线程502,则业务线程502在接收到数据访问请求之后向卸载卡发送该数据访问请求,卸载卡在接收到该数据访问请求之后将该数据访问请求添加至该业务线程502对应的缓存队列(具体为缓存队列505)。然后卸载卡从缓存队列505中读取该数据访问请求,并且根据读取的该数据访问请求中包括的业务线程502的标识以及目标存储区域的标识从标识连接关系中确定出业务线程502与目标存储区域之间的链路(假设为QP对509表征的链路)。最终卸载卡通过该QP对509表征的链路向第二云存储服务器发送数据访问请求。Assuming that the source service thread in the above-mentioned client is service thread 502, service thread 502 sends the data access request to the unloading card after receiving the data access request, and the unloading card adds the data access request to the cache queue (specifically cache queue 505) corresponding to the service thread 502 after receiving the data access request. Then the unloading card reads the data access request from the cache queue 505, and determines the link between the service thread 502 and the target storage area from the identification connection relationship according to the identification of the service thread 502 included in the read data access request and the identification of the target storage area (assuming it is the link represented by QP pair 509). Finally, the unloading card sends the data access request to the second cloud storage server through the link represented by the QP pair 509.

本申请实施例中,卸载卡在接收到客户端发送的数据访问请求之后,可以将该数据访问请求添加至与源业务线程对应的目标缓存队列。从而卸载卡可以从该目标缓存队列中读取该数据访问请求,并且根据读取的该数据访问请求中包括的源业务线程的标识以及目标存储区域的标识从标识连接关系中确定源业务线程与目标存储区域之间的链路。最终卸载卡通过该链路向第二云存储服务器发送该数据访问请求。本申请实施例中,每一个业务线程可以遵循单线程轮询,具体为一个业务线程始终将其获取到的数据访问请求发送至卸载卡(具体为卸载卡中与该业务线程对应的缓存队列)。然后卸载卡可以从每一个业务线程对应的缓存队列中读取该每一个业务线程对应的数据访问请求,从而实现相关数据的访问过程。如此卸载卡及至少一个业务线程可以组合得到RTC模型,能够降低数据访问过程中的端到端时延。In an embodiment of the present application, after receiving the data access request sent by the client, the unloading card can add the data access request to the target cache queue corresponding to the source business thread. Thus, the unloading card can read the data access request from the target cache queue, and determine the link between the source business thread and the target storage area from the identifier connection relationship according to the identifier of the source business thread included in the read data access request and the identifier of the target storage area. Finally, the unloading card sends the data access request to the second cloud storage server through the link. In an embodiment of the present application, each business thread can follow single-threaded polling, specifically a business thread always sends the data access request it obtains to the unloading card (specifically the cache queue corresponding to the business thread in the unloading card). Then the unloading card can read the data access request corresponding to each business thread from the cache queue corresponding to each business thread, thereby realizing the access process of related data. In this way, the unloading card and at least one business thread can be combined to obtain an RTC model, which can reduce the end-to-end delay in the data access process.

可选地,上述卸载卡还包括网络处理单元(network process unit,NPU),该NPU用于将上 述数据访问请求添加至目标缓存队列并且从目标缓存队列中读取数据访问请求。Optionally, the above-mentioned offload card further includes a network processing unit (NPU), which is used to The data access request is added to the target cache queue and the data access request is read from the target cache queue.

在本申请实施例的一种实现方式中,卸载卡可以对上述至少一个业务线程中的每一个业务线程进行实时检测。当检测到该至少一个业务线程中的任一个业务线程(例如源业务线程)中断时,卸载卡可以对该源业务线程执行重建操作。In one implementation of the embodiment of the present application, the unloading card can perform real-time detection on each of the at least one service thread. When it is detected that any service thread (e.g., source service thread) of the at least one service thread is interrupted, the unloading card can perform a reconstruction operation on the source service thread.

具体的,重建操作是一个用于重新建链的操作,也可以理解为一个消息的保活机制。一个业务线程(或单线程)中断也可以理解为该业务线程断开或者该业务线程处于异常状态。本申请实施例中,当检测到源业务线程中断时,说明该源业务线程处于异常状态。此时卸载卡对该源业务线程执行重建操作,可以保证源业务线程持续处于正常状态,从而进一步提升数据访问的有效性。Specifically, the reconstruction operation is an operation for re-establishing the link, which can also be understood as a message keep-alive mechanism. The interruption of a business thread (or single thread) can also be understood as the disconnection of the business thread or the business thread is in an abnormal state. In the embodiment of the present application, when the interruption of the source business thread is detected, it means that the source business thread is in an abnormal state. At this time, the unload card performs a reconstruction operation on the source business thread, which can ensure that the source business thread continues to be in a normal state, thereby further improving the effectiveness of data access.

值得注意的是,卸载卡通过源业务线程与目标存储区域之间的链路向第二云存储服务器发送数据访问请求之后,卸载卡可能存在数据访问成功的情况,也可能存在数据访问失败的情况。此时第二云存储服务器可以向卸载卡返回相应的消息,以提示卸载卡对目标存储区域中的数据访问是否成功。基于此,结合图5,如图7所示,本申请实施例提供的数据访问方法还可以包括S104-S105。It is worth noting that after the unloading card sends a data access request to the second cloud storage server through the link between the source service thread and the target storage area, the unloading card may have a successful data access or a failed data access. At this time, the second cloud storage server can return a corresponding message to the unloading card to prompt the unloading card whether the data access to the target storage area is successful. Based on this, in combination with Figure 5, as shown in Figure 7, the data access method provided in the embodiment of the present application can also include S104-S105.

S104、卸载卡接收第二云存储服务器发送的访问成功响应。S104: The offloading card receives an access success response sent by the second cloud storage server.

其中,该访问成功响应用于通知该卸载卡在目标存储区域中成功对目标对象进行预设操作。The access success response is used to notify the offloading card that a preset operation is successfully performed on the target object in the target storage area.

应理解,卸载卡通过源业务线程与目标存储区域之间的链路向第二云存储服务器发送数据访问请求之后,第二云存储服务器可以确定卸载卡是否能够成功在目标存储区域中对目标对象进行预设操作。在卸载卡在目标存储区域中成功对目标对象进行预设操作的情况下,第二云存储服务器可以向卸载卡发送该访问成功响应。It should be understood that after the unloading card sends the data access request to the second cloud storage server through the link between the source service thread and the target storage area, the second cloud storage server can determine whether the unloading card can successfully perform the preset operation on the target object in the target storage area. In the case where the unloading card successfully performs the preset operation on the target object in the target storage area, the second cloud storage server can send the access success response to the unloading card.

本申请实施例中,卸载卡在目标存储区域中成功对目标对象进行预设操作也可以理解为卸载卡成功访问目标存储区域中的数据。In the embodiment of the present application, the unloading card successfully performing a preset operation on the target object in the target storage area can also be understood as the unloading card successfully accessing the data in the target storage area.

S105、卸载卡向客户端发送访问成功响应。S105: The uninstall card sends an access success response to the client.

可以理解的是,客户端(具体可以为源业务线程)在接收到卸载卡发送的访问成功响应之后,即可以确定卸载卡对在目标存储区域中成功对目标对象进行预设操作。在卸载卡访问目标存储区域中的数据前后,客户端(或源业务线程)仅通过卸载卡收发消息,无需感知每一个业务线程与每一个存储区域之间的连接关系,能够快速访问到相关业务数据,降低了数据访问过程中产生的时延。It is understandable that after receiving the access success response sent by the unloading card, the client (specifically, the source service thread) can determine that the unloading card has successfully performed the preset operation on the target object in the target storage area. Before and after the unloading card accesses the data in the target storage area, the client (or the source service thread) only sends and receives messages through the unloading card, and does not need to perceive the connection relationship between each service thread and each storage area, so it can quickly access the relevant service data, reducing the delay generated during the data access process.

在一种可选的实现方式中,在卸载卡无法在目标存储区域中对目标对象进行预设操作或者卸载卡无法访问目标存储区域中的数据的情况下,第二云存储服务器也可以向卸载卡发送访问失败响应,该访问失败响应用于通知该卸载卡无法对在该目标存储区域中对目标对象进行预设操作。此时,卸载卡也可以向客户端(具体可以为源业务线程)发送该访问失败响应。In an optional implementation, when the unloading card cannot perform a preset operation on the target object in the target storage area or the unloading card cannot access the data in the target storage area, the second cloud storage server may also send an access failure response to the unloading card, and the access failure response is used to notify the unloading card that the preset operation cannot be performed on the target object in the target storage area. At this time, the unloading card may also send the access failure response to the client (specifically, the source service thread).

可选地,上述卸载卡(或NPU)中可以包括网络数据模块和网络路由模块。具体的,网络数据模块用于接收客户端发送的数据访问请求,以及接收第二云存储服务器发送的访问成功响应。网络路由模块用于根据源业务线程的标识以及目标存储区域的标识从标识连接关系中确定源业务线程与目标存储区域之间的链路。Optionally, the above-mentioned unloading card (or NPU) may include a network data module and a network routing module. Specifically, the network data module is used to receive a data access request sent by the client, and receive an access success response sent by the second cloud storage server. The network routing module is used to determine the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area.

以下以一个完整的示例说明本申请实施例中对目标存储区域中的数据进行访问的过程:The following is a complete example to illustrate the process of accessing data in the target storage area in an embodiment of the present application:

如图8所示,本申请实施例提供一种数据访问系统,该数据访问系统包括第一云存储服务器和第二云存储服务器,该第一云存储服务器中包括客户端60和卸载卡70,客户端60包括业务线程601、业务线程602、业务线程603、业务线程604以及业务线程605,第二云存储服务器包括存储池801和存储池802。其中,存储池801包括存储区域8011、存储区域8012以及存储区域8013。存储池802包括存储区域8021、存储区域8022以及存储区域8023。As shown in FIG8 , an embodiment of the present application provides a data access system, which includes a first cloud storage server and a second cloud storage server, wherein the first cloud storage server includes a client 60 and an uninstall card 70, the client 60 includes a service thread 601, a service thread 602, a service thread 603, a service thread 604, and a service thread 605, and the second cloud storage server includes a storage pool 801 and a storage pool 802. The storage pool 801 includes a storage area 8011, a storage area 8012, and a storage area 8013. The storage pool 802 includes a storage area 8021, a storage area 8022, and a storage area 8023.

假设上述源业务线程为业务线程601,目标存储区域为存储区域8011。则数据访问过程具体为:首先,客户端60中的业务线程601在接收到数据访问请求之后向卸载卡70发送数据访问请求,该数据访问请求包括业务线程601的标识以及存储区域8011的标识。之后,卸载卡70根据业务线程601的标识以及存储区域8011的标识从标识连接关系中确定业务线程601与存储区域8011之间的链路,并通过该链路向第二云存储服务器发送数据访问请求。然后,卸载卡70接收第二云存储服务器发送的访问成功响应,该访问成功响应用于通知卸载卡70在存储区域8011中成功对目标对象进行预设操作。最终,卸载卡70向客户端60中的业务线程601发送该访问成功 响应。Assume that the above-mentioned source business thread is business thread 601, and the target storage area is storage area 8011. Then the data access process is specifically as follows: First, after receiving the data access request, the business thread 601 in the client 60 sends a data access request to the unloading card 70, and the data access request includes the identifier of the business thread 601 and the identifier of the storage area 8011. Afterwards, the unloading card 70 determines the link between the business thread 601 and the storage area 8011 from the identifier connection relationship according to the identifier of the business thread 601 and the identifier of the storage area 8011, and sends the data access request to the second cloud storage server through the link. Then, the unloading card 70 receives the access success response sent by the second cloud storage server, and the access success response is used to notify the unloading card 70 that the preset operation on the target object in the storage area 8011 is successfully performed. Finally, the unloading card 70 sends the access success response to the business thread 601 in the client 60. response.

本申请还提供一种基于云存储服务的卸载卡,该云存储服务运行在位于多个区域中的一个区域的至少一个云数据中心的至少两个云存储服务器上,该至少两个云存储服务器中的第一云存储服务器上插置有该卸载卡并且安装有客户端。如图9所示,卸载卡90包括:接收模块901、确定模块902以及发送模块903。The present application also provides an offload card based on a cloud storage service, the cloud storage service running on at least two cloud storage servers located in at least one cloud data center in one of the multiple regions, the offload card being inserted in a first cloud storage server of the at least two cloud storage servers and a client being installed. As shown in FIG9 , the offload card 90 comprises: a receiving module 901 , a determining module 902 and a sending module 903 .

接收模块901,用于接收所述客户端发送的数据访问请求,所述数据访问请求包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,所述源业务线程为所述客户端中包括的至少一个业务线程中的任一个,所述数据访问请求用于指示在所述目标存储区域中对所述目标对象执行预设操作,所述预设操作包括以下的一项或几项操作的组合:读操作或者写操作,所述第二云存储服务器为所述至少两个云存储服务器中除所述第一云存储服务器以外的云储存服务器。The receiving module 901 is used to receive a data access request sent by the client, wherein the data access request includes an identifier of a source business thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object. The source business thread is any one of at least one business thread included in the client. The data access request is used to indicate that a preset operation is performed on the target object in the target storage area. The preset operation includes one or a combination of the following operations: a read operation or a write operation. The second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers.

确定模块902,用于根据所述源业务线程的标识以及所述目标存储区域的标识从标识连接关系中确定所述源业务线程与所述目标存储区域之间的链路,所述标识连接关系包括多个标识组合以及所述多个标识组合中每个标识组合对应的链路连接信息,其中,所述标识组合包括业务线程标识和存储区域标识。Determination module 902 is used to determine the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, wherein the identification connection relationship includes multiple identification combinations and link connection information corresponding to each identification combination in the multiple identification combinations, wherein the identification combination includes a business thread identification and a storage area identification.

发送模块903,用于通过所述源业务线程与所述目标存储区域之间的链路向所述第二云存储服务器发送所述数据访问请求。The sending module 903 is used to send the data access request to the second cloud storage server through the link between the source service thread and the target storage area.

可选地,确定模块902,具体用于基于该源业务线程的标识以及该目标存储区域的标识查询标识连接关系,从该标识连接关系中确定出与该源业务线程的标识以及该目标存储区域的标识匹配的目标标识组合。Optionally, the determination module 902 is specifically used to query the identifier connection relationship based on the identifier of the source business thread and the identifier of the target storage area, and determine the target identifier combination that matches the identifier of the source business thread and the identifier of the target storage area from the identifier connection relationship.

确定模块902,还具体用于将该目标标识组合对应的链路连接信息,确定为目标链路连接信息。The determination module 902 is further specifically configured to determine the link connection information corresponding to the target identifier combination as the target link connection information.

确定模块902,还具体用于将该目标链路连接信息表征的链路,确定为该源业务线程与该目标存储区域之间的链路。The determination module 902 is further specifically configured to determine the link represented by the target link connection information as the link between the source service thread and the target storage area.

可选地,一个链路连接信息为一个QP对。Optionally, one link connection information is a QP pair.

可选地,上述卸载卡90还包括至少一个缓存队列,该数据访问请求为目标缓存队列中包括的请求,该目标缓存队列为该至少一个缓存队列中与该源业务线程对应的缓存队列。Optionally, the above-mentioned offload card 90 further includes at least one cache queue, the data access request is a request included in a target cache queue, and the target cache queue is a cache queue in the at least one cache queue corresponding to the source service thread.

可选地,上述卸载卡90还包括重建模块904。Optionally, the above-mentioned unloading card 90 also includes a reconstruction module 904 .

重建模块904,用于当检测到该源业务线程中断时,对该源业务线程执行重建操作。The reconstruction module 904 is used to perform a reconstruction operation on the source service thread when it is detected that the source service thread is interrupted.

可选地,接收模块901,还用于接收该第二云存储服务器发送的访问成功响应,该访问成功响应用于通知该卸载卡在该目标存储区域中成功对该目标对象进行该预设操作。Optionally, the receiving module 901 is further used to receive an access success response sent by the second cloud storage server, where the access success response is used to notify the unloading card that the preset operation is successfully performed on the target object in the target storage area.

发送模块903,还用于向该客户端发送该访问成功响应。The sending module 903 is further configured to send the access success response to the client.

可选地,上述卸载卡90还包括添加模块905。Optionally, the above-mentioned unloading card 90 also includes an adding module 905 .

添加模块905,用于将该数据访问请求添加至目标缓存队列,该目标缓存队列为该至少一个缓存队列中与该源业务线程对应的缓存队列。The adding module 905 is used to add the data access request to a target cache queue, where the target cache queue is a cache queue in the at least one cache queue that corresponds to the source service thread.

可选地,上述卸载卡还包括读取模块906。Optionally, the above-mentioned unloading card also includes a reading module 906 .

读取模块906,用于从该目标缓存队列中读取该数据访问请求。The reading module 906 is used to read the data access request from the target cache queue.

确定模块902,具体用于根据读取的该数据访问请求中包括的该源业务线程的标识以及该目标存储区域的标识确定该链路。The determination module 902 is specifically configured to determine the link according to the identifier of the source service thread and the identifier of the target storage area included in the read data access request.

其中,接收模块901、确定模块902、发送模块903、重建模块904、添加模块905以及读取模块906以及均可以通过软件实现,或者可以通过硬件实现。示例性的,接下来以确定模块902为例,介绍确定模块902的实现方式。类似的,接收模块901、发送模块903、重建模块904、添加模块905以及读取模块906的实现方式可以参考确定模块902的实现方式。Among them, the receiving module 901, the determining module 902, the sending module 903, the reconstruction module 904, the adding module 905 and the reading module 906 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the determining module 902 is described below by taking the determining module 902 as an example. Similarly, the implementation of the receiving module 901, the sending module 903, the reconstruction module 904, the adding module 905 and the reading module 906 can refer to the implementation of the determining module 902.

模块作为软件功能单元的一种举例,确定模块902可以包括运行在计算实例上的代码。其中,计算实例可以包括物理主机(计算设备)、虚拟机、容器中的至少一种。进一步地,上述计算实例可以是一台或者多台。例如,确定模块可以包括运行在多个主机/虚拟机/容器上的代码。需要说明的是,用于运行该代码的多个主机/虚拟机/容器可以分布在相同的区域(region)中,也可以分 布在不同的region中。进一步地,用于运行该代码的多个主机/虚拟机/容器可以分布在相同的可用区(availability zone,AZ)中,也可以分布在不同的AZ中,每个AZ包括一个数据中心或多个地理位置相近的数据中心。其中,通常一个region可以包括多个AZ。As an example of a software functional unit, the determination module 902 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the determination module may include code running on multiple hosts/virtual machines/containers. It should be noted that the multiple hosts/virtual machines/containers used to run the code may be distributed in the same region or in different regions. Distributed in different regions. Furthermore, multiple hosts/virtual machines/containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region can include multiple AZs.

同样,用于运行该代码的多个主机/虚拟机/容器可以分布在同一个虚拟私有云(virtual private cloud,VPC)中,也可以分布在多个VPC中。其中,通常一个VPC设置在一个region内,同一region内两个VPC之间,以及不同region的VPC之间跨区通信需在每个VPC内设置通信网关,经通信网关实现VPC之间的互连。Similarly, multiple hosts/virtual machines/containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

模块作为硬件功能单元的一种举例,确定模块902可以包括至少一个计算设备,如服务器等。或者,确定模块902也可以是利用专用集成电路(application-specific integrated circuit,ASIC)实现、或可编程逻辑器件(programmable logic device,PLD)实现的设备等。其中,上述PLD可以是复杂程序逻辑器件(complex programmable logical device,CPLD)、现场可编程门阵列(field-programmable gate array,FPGA)、通用阵列逻辑(generic array logic,GAL)或其任意组合实现。As an example of a hardware functional unit, the determination module 902 may include at least one computing device, such as a server, etc. Alternatively, the determination module 902 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

确定模块902包括的多个计算设备可以分布在相同的region中,也可以分布在不同的region中。确定模块902包括的多个计算设备可以分布在相同的AZ中,也可以分布在不同的AZ中。同样,确定模块902包括的多个计算设备可以分布在同一个VPC中,也可以分布在多个VPC中。其中,所述多个计算设备可以是服务器、ASIC、PLD、CPLD、FPGA和GAL等计算设备的任意组合。The multiple computing devices included in the determination module 902 may be distributed in the same region or in different regions. The multiple computing devices included in the determination module 902 may be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the determination module 902 may be distributed in the same VPC or in multiple VPCs. The multiple computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

需要说明的是,在其他实施例中,确定模块902可以用于执行数据访问方法中的任意步骤,接收模块901可以用于执行数据访问方法中的任意步骤,发送模块903可以用于执行数据访问方法中的任意步骤,重建模块904可以用于执行数据访问方法中的任意步骤,添加模块905可以用于执行数据访问方法中的任意步骤,读取模块906也可以用于执行数据访问方法中的任意步骤。接收模块901、确定模块902、发送模块903、重建模块904、添加模块905以及读取模块906负责实现的步骤可根据需要指定,通过接收模块901、确定模块902、发送模块903、重建模块904、添加模块905以及读取模块906分别实现数据访问方法中不同的步骤来实现卸载卡的全部功能。It should be noted that, in other embodiments, the determination module 902 can be used to execute any step in the data access method, the receiving module 901 can be used to execute any step in the data access method, the sending module 903 can be used to execute any step in the data access method, the reconstruction module 904 can be used to execute any step in the data access method, the adding module 905 can be used to execute any step in the data access method, and the reading module 906 can also be used to execute any step in the data access method. The steps that the receiving module 901, the determination module 902, the sending module 903, the reconstruction module 904, the adding module 905 and the reading module 906 are responsible for implementing can be specified as needed, and the receiving module 901, the determination module 902, the sending module 903, the reconstruction module 904, the adding module 905 and the reading module 906 respectively implement different steps in the data access method to realize all the functions of the unloading card.

本申请还提供一种计算设备1000。如图10所示,计算设备1000包括:总线1002、处理器1004、存储器1006和通信接口1008。处理器1004、存储器1006和通信接口1008之间通过总线1002通信。计算设备1000可以是服务器或终端设备。应理解,本申请不限定计算设备1000中的处理器、存储器的个数。The present application also provides a computing device 1000. As shown in FIG10 , the computing device 1000 includes: a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008. The processor 1004, the memory 1006, and the communication interface 1008 communicate with each other through the bus 1002. The computing device 1000 may be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1000.

总线1002可以是外设部件互连标准(peripheral component interconnect,PCI)总线或扩展工业标准结构(extended industry standard architecture,EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示,图10中仅用一条线表示,但并不表示仅有一根总线或一种类型的总线。总线1004可包括在计算设备1000各个部件(例如,存储器1006、处理器1004、通信接口1008)之间传送信息的通路。The bus 1002 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, FIG. 10 is represented by only one line, but does not mean that there is only one bus or one type of bus. The bus 1004 may include a path for transmitting information between various components of the computing device 1000 (e.g., the memory 1006, the processor 1004, and the communication interface 1008).

处理器1004可以包括中央处理器(central processing unit,CPU)、图形处理器(graphics processing unit,GPU)、微处理器(micro processor,MP)或者数字信号处理器(digital signal processor,DSP)等处理器中的任意一种或多种。Processor 1004 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP).

存储器1006可以包括易失性存储器(volatile memory),例如随机存取存储器(random access memory,RAM)。处理器1004还可以包括非易失性存储器(non-volatile memory),例如只读存储器(read-only memory,ROM),快闪存储器,机械硬盘(hard disk drive,HDD)或固态硬盘(solid state drive,SSD)。The memory 1006 may include a volatile memory (volatile memory), such as a random access memory (RAM). The processor 1004 may also include a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

存储器1006中存储有可执行的程序代码,处理器1004执行该可执行的程序代码以分别实现前述接收模块、确定模块、发送模块、重建模块、添加模块以及读取模块的功能,从而实现数据访问方法。也即,存储器1006上存有用于执行数据访问方法的指令。The memory 1006 stores executable program codes, and the processor 1004 executes the executable program codes to respectively implement the functions of the aforementioned receiving module, determining module, sending module, reconstruction module, adding module, and reading module, thereby implementing the data access method. That is, the memory 1006 stores instructions for executing the data access method.

通信接口1003使用例如但不限于网络接口卡、收发器一类的收发模块,来实现计算设备1000与其他设备或通信网络之间的通信。The communication interface 1003 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1000 and other devices or a communication network.

本申请实施例还提供了一种计算设备集群。该计算设备集群包括至少一台计算设备。该计算 设备可以是服务器,例如是中心服务器、边缘服务器,或者是本地数据中心中的本地服务器。在一些实施例中,计算设备也可以是台式机、笔记本电脑或者智能手机等终端设备。The present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a notebook computer, or a smart phone.

如图11所示,所述计算设备集群包括至少一个计算设备1000。计算设备集群中的一个或多个计算设备1000中的存储器1006中可以存有相同的用于执行数据访问方法的指令。As shown in Fig. 11, the computing device cluster includes at least one computing device 1000. The memory 1006 in one or more computing devices 1000 in the computing device cluster may store the same instructions for executing the data access method.

在一些可能的实现方式中,该计算设备集群中的一个或多个计算设备1000的存储器1006中也可以分别存有用于执行数据访问方法的部分指令。换言之,一个或多个计算设备1000的组合可以共同执行用于执行数据访问方法的指令。In some possible implementations, the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the data access method. In other words, the combination of one or more computing devices 1000 may jointly execute instructions for executing the data access method.

需要说明的是,计算设备集群中的不同的计算设备1000中的存储器1006可以存储不同的指令,分别用于执行数据访问装置的部分功能。也即,不同的计算设备1000中的存储器1006存储的指令可以实现接收模块、确定模块、发送模块、添加模块、读取模块以及重建模块中的一个或多个模块的功能。It should be noted that the memory 1006 in different computing devices 1000 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the data access device. That is, the instructions stored in the memory 1006 in different computing devices 1000 can implement the functions of one or more modules of the receiving module, the determining module, the sending module, the adding module, the reading module and the reconstruction module.

在一些可能的实现方式中,计算设备集群中的一个或多个计算设备可以通过网络连接。其中,所述网络可以是广域网或局域网等等。图12示出了一种可能的实现方式。如图12所示,两个计算设备1000A和1000B之间通过网络进行连接。具体地,通过各个计算设备中的通信接口与所述网络进行连接。在这一类可能的实现方式中,计算设备1000A中的存储器1006中存有执行确定模块、添加模块、读取模块以及重建模块的功能的指令。同时,计算设备1000B中的存储器1006中存有执行接收模块和发送模块的功能的指令。In some possible implementations, one or more computing devices in the computing device cluster can be connected via a network. Wherein, the network can be a wide area network or a local area network, etc. Figure 12 shows a possible implementation. As shown in Figure 12, two computing devices 1000A and 1000B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1006 in the computing device 1000A stores instructions for executing the functions of the determination module, the addition module, the reading module, and the reconstruction module. At the same time, the memory 1006 in the computing device 1000B stores instructions for executing the functions of the receiving module and the sending module.

图12所示的计算设备集群之间的连接方式可以是考虑到本申请提供的数据访问方法需要大量接收过程和发送过程。因此考虑将接收模块和发送模块实现的功能交由计算设备1000B执行。The connection mode between the computing device clusters shown in FIG12 may be considered that the data access method provided by the present application requires a large number of receiving processes and sending processes, so it is considered that the functions implemented by the receiving module and the sending module are handed over to the computing device 1000B for execution.

应理解,图12中示出的计算设备1000A的功能也可以由多个计算设备1000完成。同样,计算设备1000B的功能也可以由多个计算设备1000完成。It should be understood that the functions of the computing device 1000A shown in FIG12 may also be completed by multiple computing devices 1000. Similarly, the functions of the computing device 1000B may also be completed by multiple computing devices 1000.

本申请实施例还提供了另一种计算设备集群。该计算设备集群中各计算设备之间的连接关系可以类似的参考图11和图12所述计算设备集群的连接方式。不同的是,该计算设备集群中的一个或多个计算设备1000中的存储器1006中可以存有相同的用于执行数据访问方法的指令。The embodiment of the present application also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to the connection mode of the computing device cluster described in Figures 11 and 12. The difference is that the memory 1006 in one or more computing devices 1000 in the computing device cluster can store the same instructions for executing the data access method.

在一些可能的实现方式中,该计算设备集群中的一个或多个计算设备1000的存储器1006中也可以分别存有用于执行数据访问方法的部分指令。换言之,一个或多个计算设备1000的组合可以共同执行用于执行数据访问方法的指令。In some possible implementations, the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the data access method. In other words, the combination of one or more computing devices 1000 may jointly execute instructions for executing the data access method.

本申请实施例还提供了一种包含指令的计算机程序产品。所述计算机程序产品可以是包含指令的,能够运行在计算设备上或被储存在任何可用介质中的软件或程序产品。当所述计算机程序产品在至少一个计算设备上运行时,使得至少一个计算设备执行数据访问方法。The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the data access method.

本申请实施例还提供了一种计算机可读存储介质。所述计算机可读存储介质可以是计算设备能够存储的任何可用介质或者是包含一个或多个可用介质的数据中心等数据存储设备。所述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质(例如固态硬盘)等。该计算机可读存储介质包括指令,所述指令指示计算设备执行数据访问方法。The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute a data access method.

最后应说明的是:以上实施例仅用以说明本发明的技术方案,而非对其限制;尽管参照前述实施例对本发明进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本发明各实施例技术方案的保护范围。 Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims (16)

一种基于云存储服务的数据访问方法,其特征在于,所述云存储服务运行在位于多个区域中的一个区域的至少一个云数据中心的至少两个云存储服务器上,所述至少两个云存储服务器中的第一云存储服务器上插置有卸载卡并且安装有客户端,所述方法由所述卸载卡执行,所述方法包括:A data access method based on a cloud storage service, characterized in that the cloud storage service runs on at least two cloud storage servers located in at least one cloud data center in one of multiple regions, an offload card is inserted in a first cloud storage server of the at least two cloud storage servers and a client is installed, the method is executed by the offload card, and the method includes: 接收所述客户端发送的数据访问请求,所述数据访问请求包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,所述源业务线程为所述客户端中包括的至少一个业务线程中的任一个,所述数据访问请求用于指示在所述目标存储区域中对所述目标对象执行预设操作,所述预设操作包括以下的一项或几项操作的组合:读操作或者写操作,所述第二云存储服务器为所述至少两个云存储服务器中除所述第一云存储服务器以外的云储存服务器;receiving a data access request sent by the client, the data access request including an identifier of a source service thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object, the source service thread being any one of at least one service thread included in the client, the data access request being used to instruct to perform a preset operation on the target object in the target storage area, the preset operation including one or a combination of the following operations: a read operation or a write operation, the second cloud storage server being a cloud storage server other than the first cloud storage server among the at least two cloud storage servers; 根据所述源业务线程的标识以及所述目标存储区域的标识从标识连接关系中确定所述源业务线程与所述目标存储区域之间的链路,所述标识连接关系包括多个标识组合以及所述多个标识组合中每个标识组合对应的链路连接信息,其中,所述标识组合包括业务线程标识和存储区域标识;Determine a link between the source business thread and the target storage area from an identification connection relationship according to the identification of the source business thread and the identification of the target storage area, wherein the identification connection relationship includes a plurality of identification combinations and link connection information corresponding to each of the plurality of identification combinations, wherein the identification combination includes a business thread identification and a storage area identification; 通过所述源业务线程与所述目标存储区域之间的链路向所述第二云存储服务器发送所述数据访问请求。The data access request is sent to the second cloud storage server through a link between the source service thread and the target storage area. 根据权利要求1所述的数据访问方法,其特征在于,所述根据所述源业务线程的标识以及所述目标存储区域的标识从标识连接关系中确定所述源业务线程与所述目标存储区域之间的链路,包括:The data access method according to claim 1, characterized in that the step of determining the link between the source service thread and the target storage area from an identification connection relationship according to the identification of the source service thread and the identification of the target storage area comprises: 基于所述源业务线程的标识以及所述目标存储区域的标识查询所述标识连接关系,从所述标识连接关系中确定出与所述源业务线程的标识以及所述目标存储区域的标识匹配的目标标识组合;querying the identifier connection relationship based on the identifier of the source service thread and the identifier of the target storage area, and determining a target identifier combination matching the identifier of the source service thread and the identifier of the target storage area from the identifier connection relationship; 将所述目标标识组合对应的链路连接信息,确定为目标链路连接信息;Determine the link connection information corresponding to the target identifier combination as the target link connection information; 将所述目标链路连接信息表征的链路,确定为所述源业务线程与所述目标存储区域之间的链路。The link represented by the target link connection information is determined as a link between the source service thread and the target storage area. 根据权利要求1或2所述的数据访问方法,其特征在于,一个链路连接信息为一个队列QP对。The data access method according to claim 1 or 2 is characterized in that one link connection information is a queue QP pair. 根据权利要求1-3中任一项所述的数据访问方法,其特征在于,所述卸载卡包括至少一个缓存队列,所述数据访问请求为目标缓存队列中包括的请求,所述目标缓存队列为所述至少一个缓存队列中与所述源业务线程对应的缓存队列。The data access method according to any one of claims 1-3 is characterized in that the unloading card includes at least one cache queue, the data access request is a request included in a target cache queue, and the target cache queue is a cache queue in the at least one cache queue corresponding to the source business thread. 根据权利要求1-4中任一项所述的数据访问方法,其特征在于,所述方法还包括:The data access method according to any one of claims 1 to 4, characterized in that the method further comprises: 当检测到所述源业务线程中断时,对所述源业务线程执行重建操作。When the interruption of the source service thread is detected, a reconstruction operation is performed on the source service thread. 根据权利要求1-5中任一项所述的数据访问方法,其特征在于,所述方法还包括:The data access method according to any one of claims 1 to 5, characterized in that the method further comprises: 接收所述第二云存储服务器发送的访问成功响应,所述访问成功响应用于通知所述卸载卡在所述目标存储区域中成功对所述目标对象进行所述预设操作;receiving an access success response sent by the second cloud storage server, wherein the access success response is used to notify the offloading card that the preset operation is successfully performed on the target object in the target storage area; 向所述客户端发送所述访问成功响应。Sending the access success response to the client. 一种基于云存储服务的卸载卡,其特征在于,所述云存储服务运行在位于多个区域中的一个区域的至少一个云数据中心的至少两个云存储服务器上,所述至少两个云存储服务器中的第一云存储服务器上插置有所述卸载卡并且安装有客户端,所述卸载卡包括:接收模块、确定模块以及发送模块;An offloading card based on a cloud storage service, characterized in that the cloud storage service runs on at least two cloud storage servers located in at least one cloud data center in one of multiple regions, the offloading card is inserted in a first cloud storage server of the at least two cloud storage servers and a client is installed, and the offloading card comprises: a receiving module, a determining module and a sending module; 所述接收模块,用于接收所述客户端发送的数据访问请求,所述数据访问请求包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,所述源业务线程为所述客户端中包括的至少一个业务线程中的任一个,所述数据访问请求用于指示在所述目标存储区域中对所述目标对象执行预设操作,所述预设操作包括以下的一项或几项操作的组合:读操作或者写操作,所述第二云存储服务器为所述至少两个云存储服务器中除所述第一云存储服务器以外的云储存服务器;The receiving module is used to receive a data access request sent by the client, the data access request includes an identifier of a source business thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object, the source business thread is any one of at least one business thread included in the client, the data access request is used to instruct to perform a preset operation on the target object in the target storage area, the preset operation includes one or a combination of the following operations: a read operation or a write operation, the second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers; 所述确定模块,用于根据所述源业务线程的标识以及所述目标存储区域的标识从标识连接关系中确定所述源业务线程与所述目标存储区域之间的链路,所述标识连接关系包括多个标识组合以及所述多个标识组合中每个标识组合对应的链路连接信息,其中,所述标识组合包括业务线程 标识和存储区域标识;The determination module is used to determine the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, wherein the identification connection relationship includes multiple identification combinations and link connection information corresponding to each identification combination in the multiple identification combinations, wherein the identification combination includes the business thread identification and storage area identification; 所述发送模块,用于通过所述源业务线程与所述目标存储区域之间的链路向所述第二云存储服务器发送所述数据访问请求。The sending module is used to send the data access request to the second cloud storage server through the link between the source service thread and the target storage area. 根据权利要求7所述的卸载卡,其特征在于,The uninstall card according to claim 7, characterized in that: 所述确定模块,具体用于基于所述源业务线程的标识以及所述目标存储区域的标识查询标识连接关系,从所述标识连接关系中确定出与所述源业务线程的标识以及所述目标存储区域的标识匹配的目标标识组合;The determination module is specifically used to query the identifier connection relationship based on the identifier of the source service thread and the identifier of the target storage area, and determine the target identifier combination matching the identifier of the source service thread and the identifier of the target storage area from the identifier connection relationship; 所述确定模块,还具体用于将所述目标标识组合对应的链路连接信息,确定为目标链路连接信息;The determination module is further specifically configured to determine the link connection information corresponding to the target identifier combination as the target link connection information; 所述确定模块,还具体用于将所述目标链路连接信息表征的链路,确定为所述源业务线程与所述目标存储区域之间的链路。The determining module is further specifically configured to determine the link represented by the target link connection information as the link between the source service thread and the target storage area. 根据权利要求7或8所述的卸载卡,其特征在于,一个链路连接信息为一个队列QP对。The unloading card according to claim 7 or 8 is characterized in that one link connection information is a queue QP pair. 根据权利要求7-9中任一项所述的卸载卡,其特征在于,所述卸载卡还包括至少一个缓存队列,所述数据访问请求为目标缓存队列中包括的请求,所述目标缓存队列为所述至少一个缓存队列中与所述源业务线程对应的缓存队列。The unloading card according to any one of claims 7-9 is characterized in that the unloading card also includes at least one cache queue, the data access request is a request included in a target cache queue, and the target cache queue is a cache queue in the at least one cache queue corresponding to the source business thread. 根据权利要求7-10中任一项所述的卸载卡,其特征在于,所述卸载卡还包括重建模块;The unloading card according to any one of claims 7 to 10, characterized in that the unloading card further comprises a reconstruction module; 所述重建模块,用于当检测到所述源业务线程中断时,对所述源业务线程执行重建操作。The reconstruction module is used to perform a reconstruction operation on the source service thread when it is detected that the source service thread is interrupted. 根据权利要求7-11中任一项所述的卸载卡,其特征在于,The unloading card according to any one of claims 7 to 11, characterized in that: 所述接收模块,还用于接收所述第二云存储服务器发送的访问成功响应,所述访问成功响应用于通知所述卸载卡在所述目标存储区域中成功对所述目标对象进行所述预设操作;The receiving module is further used to receive an access success response sent by the second cloud storage server, wherein the access success response is used to notify the unloading card that the preset operation is successfully performed on the target object in the target storage area; 所述发送模块,还用于向所述客户端发送所述访问成功响应。The sending module is further used to send the access success response to the client. 一种基于云存储服务的数据访问系统,其特征在于,所述云存储服务运行在位于多个区域中的一个区域的至少一个云数据中心的至少两个云存储服务器上,所述数据访问系统包括所述至少两个云存储服务器,所述至少两个云存储服务器中的第一云存储服务器上插置有卸载卡并且安装有客户端;A data access system based on a cloud storage service, characterized in that the cloud storage service runs on at least two cloud storage servers located in at least one cloud data center in one of multiple regions, the data access system comprises the at least two cloud storage servers, and an offload card is inserted in a first cloud storage server of the at least two cloud storage servers and a client is installed; 所述客户端向所述卸载卡发送数据访问请求,所述数据访问请求包括源业务线程的标识、第二云存储服务器中目标存储区域的标识以及目标对象的标识,所述源业务线程为所述客户端中包括的至少一个业务线程中的任一个,所述数据访问请求用于指示在所述目标存储区域中对所述目标对象执行预设操作,所述预设操作包括以下的一项或几项操作的组合:读操作或者写操作,所述第二云存储服务器为所述至少两个云存储服务器中除所述第一云存储服务器以外的云储存服务器;The client sends a data access request to the offload card, the data access request includes an identifier of a source service thread, an identifier of a target storage area in a second cloud storage server, and an identifier of a target object, the source service thread is any one of at least one service thread included in the client, the data access request is used to instruct to perform a preset operation on the target object in the target storage area, the preset operation includes one or a combination of the following operations: a read operation or a write operation, the second cloud storage server is a cloud storage server other than the first cloud storage server among the at least two cloud storage servers; 所述卸载卡根据所述源业务线程的标识以及所述目标存储区域的标识从标识连接关系中确定所述源业务线程与所述目标存储区域之间的链路,并且通过所述源业务线程与所述目标存储区域之间的链路向所述第二云存储服务器发送所述数据访问请求,所述标识连接关系包括多个标识组合以及所述多个标识组合中每个标识组合对应的链路连接信息,其中,所述标识组合包括业务线程标识和存储区域标识。The unloading card determines the link between the source business thread and the target storage area from the identification connection relationship according to the identification of the source business thread and the identification of the target storage area, and sends the data access request to the second cloud storage server through the link between the source business thread and the target storage area, wherein the identification connection relationship includes multiple identification combinations and link connection information corresponding to each identification combination in the multiple identification combinations, wherein the identification combination includes a business thread identification and a storage area identification. 一种计算设备集群,其特征在于,包括至少一个计算设备,每个计算设备包括处理器和存储器;A computing device cluster, characterized in that it includes at least one computing device, each computing device includes a processor and a memory; 所述至少一个计算设备的处理器用于执行所述至少一个计算设备的存储器中存储的指令,以使得所述计算设备集群执行如权利要求1-6中任一项所述的数据访问方法。The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the data access method according to any one of claims 1 to 6. 一种包含指令的计算机程序产品,其特征在于,当所述指令被计算设备集群运行时,使得所述计算设备集群执行如权利要求的1-6中任一项所述的数据访问方法。A computer program product comprising instructions, characterized in that when the instructions are executed by a computing device cluster, the computing device cluster executes the data access method as described in any one of claims 1 to 6. 一种计算机可读存储介质,其特征在于,包括计算机程序指令,当所述计算机程序指令由计算设备集群执行时,所述计算设备集群执行如权利要求1-6中任一项所述的数据访问方法。 A computer-readable storage medium, characterized in that it includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the data access method according to any one of claims 1 to 6.
PCT/CN2024/087588 2023-08-01 2024-04-12 Data access method based on cloud storage service, and offload card, system and device Pending WO2025025678A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202310960462.7 2023-08-01
CN202310960462 2023-08-01
CN202311687305.XA CN119440868A (en) 2023-08-01 2023-12-08 Data access method, offloading card, system and device based on cloud storage service
CN202311687305.X 2023-12-08

Publications (1)

Publication Number Publication Date
WO2025025678A1 true WO2025025678A1 (en) 2025-02-06

Family

ID=94393209

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/087588 Pending WO2025025678A1 (en) 2023-08-01 2024-04-12 Data access method based on cloud storage service, and offload card, system and device

Country Status (1)

Country Link
WO (1) WO2025025678A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7971236B1 (en) * 2008-10-29 2011-06-28 Netapp, Inc. Method and system for secure remote direct memory access
CN113900972A (en) * 2020-07-06 2022-01-07 华为技术有限公司 Data transmission method, chip and equipment
CN114691026A (en) * 2020-12-31 2022-07-01 华为技术有限公司 Data access method and related equipment

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7971236B1 (en) * 2008-10-29 2011-06-28 Netapp, Inc. Method and system for secure remote direct memory access
CN113900972A (en) * 2020-07-06 2022-01-07 华为技术有限公司 Data transmission method, chip and equipment
CN114691026A (en) * 2020-12-31 2022-07-01 华为技术有限公司 Data access method and related equipment

Similar Documents

Publication Publication Date Title
CN112596960B (en) Distributed storage service switching method and device
WO2022143774A1 (en) Data access method and related device
US20230152978A1 (en) Data Access Method and Related Device
WO2022228485A1 (en) Data transmission method, data processing method, and related product
WO2018058998A1 (en) Data loading method, terminal and computing cluster
CN118606079B (en) A communication method and system based on socket interface
WO2023241676A1 (en) Memory pool operation method and apparatus
CN106936931A (en) Distributed lock implementation method, related equipment and system
CN120321250A (en) A copy transmission method, device, equipment and computer readable storage medium
KR20140070611A (en) System and method for preventing single-point bottleneck in a transactional middleware machine environment
CN115686836A (en) Unloading card provided with accelerator
WO2025025678A1 (en) Data access method based on cloud storage service, and offload card, system and device
CN114697387A (en) Data packet transmission method, device and storage medium
CN112596669A (en) Data processing method and device based on distributed storage
CN114531351A (en) Method, device and equipment for transmitting message and computer readable storage medium
WO2025007852A1 (en) Virtual-instance deployment method and system based on cloud service
CN102646058A (en) Method and device for selecting node where shared memory is located in multi-node computing system
CN106557429A (en) A kind of moving method and Node Controller of internal storage data
WO2024093958A1 (en) Access method and apparatus for storage pool
CN115185630A (en) Method and device for realizing message forwarding of openstack virtual machine
CN115695454A (en) Data storage method, device, equipment and storage medium of MEC host
CN116361026A (en) Communication method and device
CN119440868A (en) Data access method, offloading card, system and device based on cloud storage service
CN116760850B (en) Data processing method, device, equipment, medium and system
CN110300068A (en) ARP method for managing resource, device, electronic equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24847644

Country of ref document: EP

Kind code of ref document: A1