TWI753421B - Method, apparatus, and GPU node for executing an application program, and computing device and machine-readable storage medium therefor - Google Patents

Abstract

Embodiments of this specification provide methods, apparatus, and GPU nodes for executing applications. The GPU node has a server, at least one client and at least one GPU hardware. After starting and executing the application on the client, the client obtains the first version information of the API interface specified in the dynamic link library required for the execution of the application, and includes the first version information in the API command execution request and sends it to the server end. The server uses the first version information and the second version information in the local driver to perform API interface adaptation, and uses the adapted API interface to access the GPU hardware to execute API commands, and then returns the API command execution results to client.

Description

Method, apparatus and GPU node for executing application program and computing device and machine-readable storage medium thereof

Embodiments of this specification generally relate to the field of computers, and more particularly, to methods, apparatuses, and GPU nodes for executing applications.

AI (Artificial Intelligence), especially Deep Learning (DL), has been widely used in payment (face), loss assessment (image recognition), interaction and customer service (speech recognition, content filtering), etc. scene with remarkable results. Typical DL tasks require powerful computing power support. Therefore, most tasks are currently executed on GPU and other acceleration devices deployed in GPU nodes. GPU (Graphics Processing Unit, graphics processor) is a high-performance computing acceleration device. Currently widely used in AI, deep learning training and online services. In practice, the GPU hardware in the GPU node is usually replaced quickly. For example, high-performance GPU devices almost always launch a new generation of products every year, and their performance and efficiency are greatly improved. With the replacement of GPU hardware, it is necessary to install new GPU drivers and upgrade the software library. For many businesses, the verification and upgrade of the underlying software involves a wide range and is often cautious. Even DL applications (for example, GPU applications) still use the old GPU drivers and software libraries for a long time. As a result, DL applications cannot be Executes on new GPU hardware and thus does not enjoy the functional and performance improvements brought by the updated GPU hardware.

In view of the above problems, the embodiments of this specification provide a method, an apparatus and a GPU node for executing an application program. Using the method and apparatus, the application program can be executed on the new GPU hardware without modification or recompilation. According to an aspect of the embodiments of the present specification, there is provided an apparatus for executing an application program, the apparatus is applied to a server in a GPU node, and at least one GPU hardware is deployed in the GPU node, and the apparatus includes : an execution request receiving unit that receives an application program interface instruction execution request from the client, where the application program interface instruction execution request includes the first version information of the application program interface specified in the dynamic link library required for application execution, and the first version of the application program interface A version information is obtained in response to the client detecting that the application program is started and executed; the adaptation processing unit, based on the application program interface adaptation policy, according to the first version information and the second version information of the application program interface The version information is adapted to the application program interface, and the second version information is the version information of the application program interface in the driver of the at least one GPU hardware installed on the server; adapting the processed application program interface to access the at least one GPU hardware to execute the application program interface instruction; and an execution result sending unit to send the execution result of the application program interface instruction to the client. Optionally, in an example of the above aspect, the apparatus may further include: a hardware discovery unit, for discovering GPU hardware in the GPU node; and an adaptation policy creation unit, based on the discovered GPU hardware The API compatibility list creates the API adaptation policy. Optionally, in an example of the above aspect, the apparatus may further include: a GPU execution resource isolation unit, which allocates isolation resources for the GPU hardware to execute the application programming interface instructions; and an instruction priority management unit, which manages Priority of execution of the API instructions on the GPU hardware. Optionally, in an example of the above aspect, the apparatus may further include: a GPU execution optimization unit that performs execution optimization processing on the GPU hardware. Optionally, in an example of the above aspect, the client and the server are located in the same device, and the communication between the client and the server is implemented using an inter-process communication mechanism. Optionally, in an example of the above aspect, the client and the server are located in different devices, and the communication between the client and the server is implemented using a network protocol. Optionally, in an example of the above aspect, the client and the server are located in the same GPU node, or the client and the server are located in different GPU nodes. Optionally, in an example of the above aspect, the application program execution request includes application program scheduling information, the application program call information is used to specify the target GPU hardware that needs to be accessed when the application program is executed, and The target GPU hardware is part or all of the at least one GPU hardware. According to another aspect of the embodiments of the present specification, there is provided an apparatus for executing an application, the apparatus being applied to a client in a GPU node in which at least one GPU hardware is deployed, the apparatus comprising : a version information acquisition unit, in response to detecting that the application program is started to be executed, to acquire the first version information of the application program interface specified in the dynamic link library required for the execution of the application program; the execution request sending unit, to the GPU node The server side in the server sends an application program interface instruction execution request, and the application program interface instruction execution request includes the first version information, so as to perform the application program interface adaptation process and the application program execution process on the server side; and the execution result The receiving unit receives the execution result of the application programming interface instruction from the server, wherein the application programming interface adaptation process is based on the application programming interface adaptation rule according to the first version information and the second version information of the application programming interface. version information is performed, and the application program execution process uses an adapted application program interface to access the at least one GPU hardware to execute the application program interface instructions, and the second version information is the Version information of the application programming interface in the driver of the at least one GPU hardware installed on the server. According to another aspect of an embodiment of the present specification, there is provided a GPU node, comprising: a server including the above-mentioned apparatus for executing an application; at least one client, each client including the above-mentioned apparatus the described device for executing an application; and at least one GPU hardware. According to another aspect of the embodiments of the present specification, there is provided a method for executing an application, the method being applied to a server in a GPU node in which at least one GPU hardware is deployed, the method comprising : Receive an application program interface instruction execution request from the client, the application program interface instruction execution request includes the first version information of the application program interface specified in the dynamic link library required for the execution of the application program, and the first version information is the response Obtained when the client detects that the application is started and executed; and based on the application interface adaptation strategy, the application interface adaptation process is performed according to the first version information and the second version information of the application interface, The second version information is the version information of the application programming interface in the driver of the at least one GPU hardware installed on the server; the at least one application programming interface after the adaptation process is used to access the at least one The GPU hardware executes the application programming interface instruction; and sends the execution result of the application programming interface instruction to the client. Optionally, in an example of the above aspect, the API adaptation policy may be created based on an API compatibility list of the GPU hardware. Optionally, in an example of the above aspect, the client and the server are located in the same device, and the communication between the client and the server is implemented using an inter-process communication mechanism. Optionally, in an example of the above aspect, the client and the server are located in different devices, and the communication between the client and the server is implemented using a network protocol. According to another aspect of the embodiments of the present specification, there is provided a method for executing an application, the method being applied to a client in a GPU node in which at least one GPU hardware is deployed, the method comprising : In response to detecting that the application program is started to be executed, obtain the first version information of the application program interface specified in the dynamic link library required for the execution of the application program; send the application program interface instruction to the server in the GPU node to execute request, the application program interface instruction execution request includes the first version information, so that the application program interface adaptation process and the application program execution process are performed on the server; and the execution of the application program interface instruction is received from the server As a result, the application program interface adaptation process is performed according to the first version information and the second version information of the application program interface based on the application program interface adaptation rule, and the application program execution process is performed using the Adapting the processed API to access the at least one GPU hardware to execute the API command, and the second version information is a driver of the at least one GPU hardware installed on the server Version information for the API in . According to another aspect of the embodiments of the present specification, there is provided a computing device comprising: one or more processors, and memory coupled to the one or more processors, the memory storing instructions, when the The instructions, when executed by the one or more processors, cause the one or more processors to execute the method for executing an application as described above for the server. According to another aspect of the embodiments of the present specification, there is provided a machine-readable storage medium storing executable instructions that, when executed, cause the machine to execute the above-described application for executing an application applied to a server. program method. According to another aspect of the embodiments of the present specification, there is provided a computing device comprising: one or more processors, and memory coupled to the one or more processors, the memory storing instructions, when the The instructions, when executed by the one or more processors, cause the one or more processors to perform a method for executing an application as applied to a client as described above. According to another aspect of the embodiments of the present specification, there is provided a machine-readable storage medium storing executable instructions that, when executed, cause the machine to execute the above-described application for executing an application applied to a server. program method. Using the application execution method and device provided by the embodiments of this specification, by providing an application execution mechanism with a client-server architecture, the application program calls API instructions on the client, and the server Implement API interface adaptation and access GPU hardware through the adapted API interface to execute API instructions, thereby decoupling application calls to API instructions and accessing GPU hardware to specifically execute API instructions. In this way, GPU The application can load the existing API commands on the client side, and then complete the API interface adaptation on the server side, so that the adapted API interface can access the new GPU hardware to execute the API commands, so that the application can execute the API commands without modification. Or recompile on new GPU hardware.

The subject matter described herein will now be discussed with reference to example implementations. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and not to limit the scope of protection, applicability or examples set forth in the scope of the patent application. Changes may be made in the function and arrangement of elements discussed without departing from the scope of the embodiment content of this specification. Various examples may omit, substitute, or add various procedures or components as desired. For example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with respect to some examples may also be combined in other examples. As used herein, the term "including" and variations thereof represent open-ended terms meaning "including but not limited to". The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment." The term "another embodiment" means "at least one other embodiment." The terms "first", "second", etc. may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. The definition of a term is consistent throughout the specification unless the context clearly dictates otherwise. As used herein, a "GPU node" may be a subject with GPU processing capabilities, eg, a single GPU device or GPU system with GPU processing capabilities. Also, in this document, "application" and "GPU application" are used interchangeably, both intended to describe an application that can be executed on a GPU device. 1 shows a schematic diagram of a GPU system architecture 100 for executing applications. As shown in FIG. 1 , the GPU system architecture 100 may include GPU underlying hardware 110 , GPU hardware driver 120 , AI framework layer 130 and application layer 140 from bottom to top. In the embodiments of this specification, the GPU underlying hardware 110 may be one of nvidia enterprise-level P100, V100, T4 or consumer GTX 1080, etc. However, the GPU underlying hardware 110 is not limited to the above examples. The GPU underlying hardware 110 includes GPU hardware and GPU resources required for implementing GPU functions. Specifically, the GPU resources of each GPU underlying hardware 110 may include, for example, GPU display card memory, computing queues, computing task processing handles, and the like. In this specification, the GPU underlying hardware 110 may include one or more GPU hardware entities. The GPU hardware driver 120 is used to drive the GPU underlying hardware 110 to enable the GPU hardware entity to work. For example, the GPU hardware driver 120 contains version information of the API interface. Accordingly, an API interface lower than the version information contained in the GPU hardware driver 120 cannot access the GPU hardware entity. For example, if the version of the API interface in the GPU hardware driver 120 is CUDA10, an application calling CUDA9 cannot access the GPU hardware, and thus cannot be executed on the GPU hardware. Here, CUDA is the collective name of the SDK launched by GPU manufacturer nvidia, which has a corresponding open source interface, such as OpenCL. The hardware entities P100/P40/P4, V100 and T4 are shown in FIG. 1 . Among them, the version of the API interface in the driver of the hardware entity P100/P40/P4 is 384.111, the version of the API interface in the driver of V100 is 396.26, and the version of the API interface in the driver of T4 is 410 or 418 . In addition, the API interface version 410 or 418 in T4 is the highest, and the hardware entities P100/P40/P4, V100 and T4 can be accessed using the API interface of this version. The API interface version 396.26 in the driver of V100 is higher than the API interface version 384.111 in the driver of the hardware entity P100/P40/P4, so the API interface of version 396.26 can be used to access the hardware entity P100/P40/P4 and V100. The API interface version 384.111 in the driver of the hardware entity P100/P40/P4 is the lowest, and the API interface of this version can only access the hardware entity P100/P40/P4. The AI framework layer 130 is used to provide various API interfaces supported by the system, such as CUDA10, CUDA9, and CUDA8, etc., for use in building applications. At compile time, the AI framework layer 130 typically binds a specific dynamic link library (eg, CUDA8 or CUDA10) to generate an executable program. Then, at the execution time, when the GPU application based on the AI framework layer 130 is started, the required dynamic link library (eg CUDA8) is searched through the operating system and loaded into the cache memory. The AI framework layer 130 may include, for example, frameworks such as TensorFlow, PyTorch, Caffe2, etc. All known AI framework layers 130 support GPU execution, that is, the version of the API interface provided in the AI framework layer 130 is not lower than the driver of the GPU hardware In the case of the version of the API interface in the program, the API interface provided by the AI framework layer 130 can access the GPU hardware entity. When an application is successfully installed in the system, the installed application is stored in the application layer 140 . When the application is started to execute, it can be allowed to use the API interface of the loaded version to access the GPU hardware, so as to use the GPU resources provided by the GPU hardware for execution. In this specification, the application may be, for example, a user model. The user model may be any of the following, but is not limited to the following examples: CNN (Convolutional Neural Networks, convolutional neural network), RNN (Recurrent Neural Network, recurrent neural network), LSTM (Longshort term memory, long short term memory, long short term memory) Network), GAN (Generative Adversarial Networks, Generative Adversarial Networks) and other models. However, when the version of the loaded API interface is lower than the version of the API interface in the driver of the GPU hardware, as shown in Figure 2, when the GPU application calls CUDA9, the version of the called API interface is 384.111, It is lower than version 410/418, thus, the GPU application cannot use the called API interface to access the GPU hardware T4 for execution. In practice, the GPU hardware in the GPU node is usually replaced quickly. For example, high-performance GPU devices almost always launch a new generation of products every year, and their performance and efficiency are greatly improved. With the replacement of GPU hardware, it is necessary to install new GPU drivers and upgrade the software library. For many businesses, the verification and upgrade of the underlying software involves a wide range of aspects, and is often more cautious. Even DL applications (eg, GPU applications) still use the old GPU drivers and software libraries (dynamic link libraries) for a long time. In this case, if the GPU application cannot use the existing API interface to access the updated GPU hardware, it will not be able to enjoy the functionality and performance improvements brought by the updated GPU hardware. In view of the above, in the embodiments of this specification, an application execution mechanism with a client-server architecture is provided. In the application execution mechanism, the application program calls API instructions on the client, and the API interface adaptation is implemented on the server side, and the GPU hardware is accessed through the adapted API interface to execute the API instructions. This decouples the application's call to API commands and accesses the GPU hardware to execute the API commands specifically. In this way, the GPU application can load the existing API commands on the client side, and then complete the API interface adaptation on the server side to make the appropriate The configured API interface can access the new GPU hardware to execute API instructions, enabling applications to run on the new GPU hardware without modification or recompilation. A method, an apparatus, and a GPU node for executing an application program according to embodiments of the present specification will be described below with reference to FIGS. 3 to 11 . FIG. 3 shows a schematic diagram of the architecture of a GPU node 300 for executing applications according to an embodiment of the present specification. As shown in FIG. 3 , the GPU node 300 includes at least one client 310 (eg, clients 310 - 1 and 310 - 2 shown in FIG. 3 ), a server 320 and at least one GPU hardware 330 . Each client 310 includes an application layer 311 , an AI framework layer 312 and an application execution device 313 . When an application (eg, a GPU application) is successfully installed in the system, the installed application is stored in the application layer 311 . The AI framework layer 312 is used to provide various API interfaces supported by the system for use in building applications. More specifically, the AI framework layer 312 usually binds multiple dynamic link libraries (GPU dynamic link libraries), such as cuBlas, cuFFT, cuSparse, cuDNN, etc., and each dynamic link library specifies the supported API interface version information. The application program execution device 313 is configured to obtain the first version information of the API interface specified in the dynamic link library required for the execution of the application program when the application program is started to execute, and send the API command execution request to the server 320, the said The API instruction execution request includes the first version information, and the API instruction execution result is received from the server 320 . Specifically, after the GPU application starts to execute, the background system will load the required dynamic link library for each GPU application according to the dynamic link library version specified at compile time, for example, load libA.8.0 for GPU application 1. so, load libA.9.0.so for GPU application 2 and libA.10.0.so for GPU application 3. Then, the application execution device 313 extracts the version of the GPU link library actually loaded by the current GPU application. Specifically, the file name of the dynamic link library can be searched by scanning the program stack or the shared memory mapping area of the program (for example, the mapping file used by the linux dynamic link library into the processing address space), and the version in the file name can be extracted. information (eg, CUDA8/9/10 in the above example). Then, the extracted version information is included in the API command execution request and sent to the server 320 to execute the API command, and the API command execution result is received from the server 320 . FIG. 4 shows a block diagram of the application execution device 313 applied to the client according to an embodiment of the present specification. As shown in FIG. 4 , the application execution device 313 includes a version information acquisition unit 3131 , an execution request sending unit 3133 and an execution result receiving unit 3135 . The version information obtaining unit 3131 is configured to obtain the first version information of the API interface specified in the dynamic link library required for the execution of the application program in response to detecting that the application program is started to be executed. Next, the execution request sending unit 3133 sends an API command execution request to the server 320 in the GPU node 300, where the API command execution request includes the first version information, so that the server 320 can perform API interface adaptation processing and application program execution processing . The execution result receiving unit 3135 is configured to receive the execution result of the API instruction from the server 320 . Each client 310 communicates with the server 320 through IPC (Inter-Process Communication) or network protocol to send an API command execution request to the server 320 and receive an API command execution result from the server 320 . The server 320 is executed on the GPU hardware driver 340 . The server 320 is a long running daemon that runs in the background of the system for a long time. In the embodiment of this specification, a service instance may be deployed on a server 320, and the service instance may be packaged and executed in a docker container. The server 320 manages one or more GPU hardware entities 330 . One server 320 may correspond to multiple client entities 310 , or one GPU hardware entity 330 may correspond to multiple client entities 310 . The server 320 includes an application program execution device 321 . The application program execution device 321 is configured to, after receiving the API instruction execution request from the client 310, based on the first version information of the API interface contained in the received API instruction execution request and the API interface version in the locally installed driver information (ie, the second version information), the API interface is adapted according to the API interface adaptation policy, and then the adapted API interface is used to access the GPU hardware to execute API instructions. In addition, the server 320 also returns the execution result of the API instruction to the client 310 . FIG. 5 is a schematic structural diagram of an application executing apparatus 321 applied to a server according to an embodiment of the present specification. As shown in FIG. 5 , the application program execution device 321 includes an execution request receiving unit 3211 , an adaptation processing unit 3213 , an application program execution unit 3215 and an execution result sending unit 3217 . The execution request receiving unit 3211 is configured to receive an API instruction execution request from the client 310, where the API instruction execution request includes the first version information of the API interface specified in the dynamic link library required for application execution. Here, it should be noted that the server 320 may receive the API instruction execution request from the client 310 located in the same GPU node, or may receive the API instruction execution request from the client 310 located in a different GPU node. The adaptation processing unit 3213 is configured to perform API interface adaptation processing according to the first version information and the second version information based on the API interface adaptation policy. Here, the second version information is the version information of the API interface in the driver of the GPU hardware installed on the server 320. For example, for the GPU hardware T4, the second version information is 410 or 418. Here, the API interface adaptation strategy may be created based on the API interface compatibility list of the GPU hardware. For example, the API interface adaptation strategy may be pre-created based on the API interface compatibility list provided by the GPU hardware generation manufacturer. Alternatively, in one example, the application execution device 321 may include a hardware discovery unit 322 and an adaptation policy creation unit 323 . Hardware discovery unit 322 is configured to discover GPU hardware 330 in GPU node 300 . The adaptation strategy creation unit 323 is configured to create an API interface adaptation strategy based on the discovered API interface compatibility list of the GPU hardware 330 . FIG. 6 shows an example schematic diagram of an API interface adaptation strategy according to an embodiment of the present specification. Figure 6 shows 2 clients (client 1 and client 2) and 1 server, wherein the API interface version of client 1 is CUDA10, the API interface version of client 2 is CUDA9, and the server's API interface version is CUDA10. The API interface version is CUDA10. As shown in FIG. 6 , for API1 and API3, there is no parameter change in the API interface version of the server 320. Therefore, during the adaptation process, API1 is maintained for the API command execution requests sent by clients 1 and 2. And the parameters of API3 are unchanged. For API2, the parameters of the API interface version of the server end 320 change. Therefore, during the adaptation process, the request is executed for the API command sent by client 1, the parameters of API2 are kept unchanged, and the request sent by client 2 is The API command executes the request and performs parameter conversion for API2. For API4, the API interface version of the server 320 is abolished. Therefore, when performing the adaptation process, the request is executed for the API command sent by the client 1 without any operation, and the request is executed for the API command sent by the client 2. , ignore the execution and return the execution success message. Returning to FIG. 5 , after the API interface is adapted, the application execution unit 3215 uses the adapted API interface to access the GPU hardware to execute API instructions. Then, the execution result sending unit 3217 sends the execution result of the API instruction to the client 310 . It should be noted here that, in one example, the client 310 and the server 320 may be located in the same device. In this case, the communication between the client 310 and the server 320 may be implemented in an IPC manner (eg, UNIX socket, pipe or shared memory, etc.). In another example, the client 310 and the server 320 may also be located in different devices. In this case, a network protocol may be used to implement communication between the client 310 and the server 320, for example, a TCP (Transmission Control Protocol, Transmission Control Protocol) communication protocol, an IP communication protocol, or an RDMA (Remote Direct Memory) communication protocol may be used. Access, direct memory access) communication protocol, etc. to achieve communication. In addition, the server 320 can also receive API instruction execution requests from multiple clients 310 at the same time. In this case, the application execution device 321 may further include a GPU execution resource isolation unit 324 and a priority association unit 325 . The GPU execution resource isolation unit 324 allocates isolation resources for the GPU hardware to execute API instructions. The instruction priority management unit 325 is configured to manage the priority of the execution of API instructions on the GPU hardware. In addition, in order to optimize the GPU execution efficiency on the server 320 . The application execution device 321 may also include a GPU execution optimization unit 326 . GPU execution optimization unit 326 is configured to perform execution optimization processing on GPU hardware. The optimization process may include, for example, GPU display card memory optimization, GPU performance optimization, and/or GPU scalability optimization, and the like. In addition, when multiple GPU nodes 300 form a GPU computing cluster, the GPU computing cluster may further include a cluster scheduler. The cluster scheduler communicates with the client 310 in the GPU node 300 through the network or IPC. The cluster scheduler is responsible for the scheduling of GPU resources within the cluster. The server 320 reports GPU resources (physical GPU resources and/or virtual GPU resources) to the cluster scheduler, for example, through the device plug-in "nvidia device plugin". Makes the cluster scheduler control the allocation of all GPU resources within the cluster. Each client 310 requests GPU resources from the cluster scheduler. The cluster scheduler is responsible for scheduling and executing the allocation of all GPU resources, eg, by launching an instance on the target pod to allocate the target quasi-GPU resource to the corresponding client 310 . Specifically, the cluster scheduler may include, but is not limited to, the K8S (Kubernetes) scheduler (kube-scheduler) or the Kubemaker scheduler. In the case of a GPU computing cluster, the application execution request may further include application scheduling information, and the application invocation information is used to specify the target GPU hardware (that is, for the GPU hardware on which the application is executed). Here, the target GPU hardware may be part or all of the at least one GPU hardware. FIG. 7 shows a flowchart of a method 700 for executing an application according to an embodiment of the present specification. As shown in FIG. 7 , in step 710, in response to detecting that the application program is started to be executed, the client 310 obtains the first version information of the API interface specified in the dynamic link library required for the execution of the application program. Next, in step 720, the client 310 sends an API instruction execution request to the server 320, where the API instruction execution request includes the first version information. After receiving the API instruction execution request, in step 730, the server 320 performs API interface adaptation processing according to the first version information and the second version information based on the API interface adaptation policy. Here, the second version information is the version information of the API interface in the driver of at least one GPU hardware installed on the server 320 . Then, in step 740, the server 320 uses the adapted API interface to access the GPU hardware to execute the API command. Next, in step 750, the execution result of the API instruction is sent to the client 310, thereby completing the API instruction execution process. The application execution method and system architecture according to the embodiments of the present specification are described above with reference to FIGS. 3 to 7 . The system architecture according to the embodiments of this specification may be deployed in a bare metal environment, a container environment, or a VM virtual machine environment, as shown in FIG. 8A to FIG. 8C . It should be noted here that the client shown in FIGS. 8A to 8C is the client body of the client 310 disclosed in the embodiments of this specification, and the client body includes the application program execution device 313 but does not include the application program 311 and AI framework layer 312. If the system architecture is deployed in a bare metal environment, as shown in FIG. 8A , both the server side and the client side are executed on the host operating system (host OS) (for example, both are executed on linux). The server takes over all access to GPU resources through the GPU driver. Among them, if the client main body and the server end are in the same machine, the communication can use IPC communication; if the client main body and the server end are not in the same machine, the TCP protocol, IP protocol or RDMA protocol is used for communication. If the system architecture is deployed in a container environment, as shown in Figure 8B, the server side executes and manages GPU resources in a containerized manner. The client body (such as K8S pod) and the server are executed on the same physical machine, and the communication between the client body and the server can be implemented using IPC, (such as UNIX socket, Pipe or shmem) or network protocols. If the system architecture is deployed in a virtual machine environment, as shown in Figure 8C, GPU resources are given to a specific physical machine, and then the server or client main body is started in the VM Guest OS, which is equivalent to a bare metal environment. It can be seen that the system architecture can support deployment on bare metal, containers and virtual machines at the same time, making the deployment very flexible. FIG. 9 shows an example schematic diagram for deploying a new GPU node in a GPU computing cluster 900 according to an embodiment of the present specification. In an existing GPU computing cluster (for example, the GPU computing cluster formed by GPU node 1 and GPU node 2 of model GPU-A in Figure 9), it is necessary to deploy new GPU nodes (such as GPU nodes of model GPU-B and driver B) 3), first deploy the client 310 and the server 320 according to the embodiment of the present specification on the new GPU node 3 . The deployed server 320 supports the newer GPU hardware driver version B, and the API adaptation table is updated (ie, the API interface version of the server 320 is updated to version B). Then, add GPU node 3 to the existing GPU computing cluster, so that the server and client in GPU node 3 can communicate with the servers and clients of other GPU nodes (ie, GPU nodes 1 and 2) in the GPU computing cluster. Communicate to form a heterogeneous computing cluster with multiple GPU models coexisting. In this way, the API interface version of the newly added GPU node 3 is newer than the API interface versions in the GPU nodes 1 and 2. By deploying the client 310 and the server 320 according to the embodiment of the present specification in the GPU node 3, because The server 320 deployed in the GPU node 3 supports newer hardware and drivers, so using the API interface version extraction-negotiation-adaptation mechanism provided in this specification, the server 320 can adapt to the old version of the API and execute it on its behalf , so that existing GPU applications can be scheduled and executed on the newly added GPU node 3 without modification, breaking the original limitation that specific GPU applications must be executed on specific GPU models, and improving the flexibility of application deployment and scheduling within the cluster , for example, a GPU application instance can be started on a new GPU node (for example, encapsulated in a container), or a GPU application instance executing on other GPU nodes can be live migrated to the new GPU node (for example, for load balancing, system maintenance, etc. ) and other scenarios. In addition, when the heterogeneous cluster needs to deploy other GPU hardware (such as GPU-C), the above operation process can be repeated. According to the embodiments of this specification, by providing an API interface version extraction-negotiation-adaptation mechanism, it is possible to maintain a consistent interface for the upper (AI framework, application model, etc.) It can better abstract, encapsulate, and isolate GPU resources, thereby helping to improve GPU resource utilization, improve scheduling flexibility, support hot migration, and achieve transparent management. As above, with reference to FIGS. 3 to 9 , embodiments of a method and apparatus for executing an application program according to embodiments of the present specification are described. The above application program execution device may be implemented by hardware, software or a combination of hardware and software. FIG. 10 is a structural block diagram of a computing device 1000 applied to a client for executing an application program according to an embodiment of the present specification. As shown in FIG. 10 , computing device 1000 may include at least one processor 1010 , storage (eg, non-volatile storage) 1020 , memory 1030 , communication interface 1040 , and internal bus 1060 , and at least one processor 1010 , the storage 1020 , the memory 1030 and the communication interface 1040 are connected together via the bus bar 1060 . The at least one processor 1010 executes at least one computer-readable instruction stored or encoded in a computer-readable storage medium (ie, the aforementioned elements implemented in software). In one embodiment, computer-executable instructions are stored in the memory, which, when executed, cause at least one processor 1010 to: in response to detecting that the application program is started to be executed, obtain the dynamic link library required for the execution of the application program The first version information of the application programming interface specified in; send an application programming interface instruction execution request to the server in the GPU node, and the application programming interface instruction execution request includes the first version information to be used in the server The application program interface adaptation process and the application program execution process are performed by the server; and the execution result of the application program interface instruction is received from the server side, wherein the application program interface adaptation process is based on the application program interface adaptation strategy. the first version information and the second version information of the application programming interface, and the application program execution process is to use the adapted application programming interface to access the at least one GPU hardware to execute the application program The interface instruction, the second version information is the version information of the application program interface in the driver of the at least one GPU hardware installed on the server. It should be understood that the computer-executable instructions stored in the memory, when executed, cause the at least one processor 1010 to perform the various operations and functions described above in connection with FIGS. 3-9 in various embodiments of this specification. FIG. 11 is a structural block diagram of a computing device 1100 applied to a client for executing an application program according to an embodiment of the present specification. As shown in FIG. 11 , computing device 1100 may include at least one processor 1110 , storage (eg, non-volatile storage) 1120 , memory 1130 , communication interface 1140 , and internal bus 1160 , and at least one processor 1110 , the storage 1120 , the memory 1130 and the communication interface 1140 are connected together via the bus bar 1160 . The at least one processor 1110 executes at least one computer-readable instruction stored or encoded in a computer-readable storage medium (ie, the above-described elements implemented in software). In one embodiment, computer-executable instructions are stored in the memory that, when executed, cause at least one processor 1110 to: receive an application programming interface instruction execution request from a client, the application programming interface instruction execution request comprising an application program The first version information of the application program interface specified in the dynamic link library required for execution, the first version information is obtained in response to the client detecting that the application program is started and executed; based on the application program interface adaptation configure a strategy to perform application programming interface adaptation processing according to the first version information of the application programming interface and the second version information, the second version information is the driver of the at least one GPU hardware installed on the server version information of the API in the program; use the adapted API to access the at least one GPU hardware to execute the API instruction; and send the execution result of the API instruction to all described client. It should be understood that the computer-executable instructions stored in the memory, when executed, cause the at least one processor 1110 to perform the various operations and functions described above in connection with FIGS. 3-9 in various embodiments of this specification. According to one embodiment, a program product, eg, a non-transitory machine-readable medium, is provided. A non-transitory machine-readable medium may have instructions (ie, the above-described elements implemented in software) that, when executed by a machine, cause the machine to perform the various operations described above in connection with FIGS. 3-9 in various embodiments of this specification and function. Specifically, a system or device equipped with a readable storage medium may be provided, on which software code for implementing the functions of any of the above-described embodiments is stored, and a computer or device of the system or device may be provided. The processor reads and executes the instructions stored in the readable storage medium. In this case, the program code itself read from the readable medium can implement the functions of any one of the above-described embodiments, so the machine-readable code and the readable storage medium storing the machine-readable code constitute the present invention a part of. Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (eg, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes , non-volatile memory cards and ROMs. Alternatively, the code can be downloaded from a server computer or cloud over a communication network. It should be understood by those skilled in the art that various variations and modifications may be made to the various embodiments disclosed above without departing from the spirit of the invention. Therefore, the protection scope of the present invention should be defined by the appended claims. It should be noted that not all steps and units in the above-mentioned processes and system structure diagrams are necessary, and some steps or units may be omitted according to actual needs. The execution order of each step is not fixed and can be determined as required. The device structure described in the above embodiments may be a physical structure or a logical structure, that is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or may be implemented by multiple physical entities. Some components in separate devices are implemented together. In the above embodiments, the hardware unit or module may be implemented mechanically or electrically. For example, a hardware unit, module or processor may include permanent dedicated circuits or logic (eg, dedicated processors, FPGAs or ASICs) to perform corresponding operations. The hardware unit or processor may also include programmable logic or circuits (such as a general-purpose processor or other programmable processors), which may be temporarily set by software to perform corresponding operations. The specific implementation (mechanical, or dedicated permanent circuit, or temporarily provided circuit) can be determined based on cost and time considerations. The detailed description set forth above in connection with the accompanying drawings describes exemplary embodiments, but does not represent all embodiments that may be implemented or fall within the scope of the claims. The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not mean "preferred" or "advantage" over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described technology. However, these techniques may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the described embodiments. The above description of the disclosure is provided to enable any person of ordinary skill in the art to make or use the disclosure. Various modifications to this disclosure will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other transform. Thus, the present disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

100: GPU System Architecture 110: Bottom Hardware 120: Hard drive 130: Frame Layer 140: Application Layer 300: GPU node 310, 310-1, 310-2: Client 311, 311-1, 311-2: Application Layer 312, 312-1, 312-2: AI Framework Layer 313, 313-1, 313-2: Application Execution Device 320: Servo side 321: Application Execution Device 322: Hardware Discovery Unit 323: Adaptation strategy creation unit 324: GPU execution resource isolation unit 325: Instruction priority management unit 326: GPU execution optimization unit 330: GPU 340: Hard drive 3131: Version information acquisition unit 3133: Execute request sending unit 3135: Execution result receiving unit 3211: Execute request receiving unit 3213: Adaptation processing unit 3215: Application execution unit 3217: Execution result sending unit 710: Steps 720: Steps 730: Steps 740: Steps 750: Steps 900: Computing Cluster 1000: Computing Devices 1010: Processor 1020: Storage 1030: Memory 1040: Communication Interface 1060: Busbar 1100: Computing Devices 1110: Processor 1120: Storage 1130: Memory 1140: Communication interface 1160: Busbar

A further understanding of the nature and advantages of the content of the embodiments of this specification may be realized by reference to the following drawings. In the drawings, similar components or features may have the same reference numerals. [Fig. 1] shows a schematic diagram of an existing application execution architecture; [Figure 2] shows a schematic diagram of the execution status of the application under different GPU hardware; [ FIG. 3 ] shows a schematic diagram of the architecture of a GPU node for executing an application according to an embodiment of the present specification; [ Fig. 4 ] shows a block diagram of an application execution device applied to a client according to an embodiment of the present specification; [ Fig. 5 ] shows a schematic structural diagram of an application program execution device applied to a server according to an embodiment of the present specification; [Fig. 6] shows an example schematic diagram of an API interface adaptation strategy according to an embodiment of the present specification; [ FIG. 7 ] A flowchart illustrating a method for executing an application according to an embodiment of the present specification; [ FIG. 8A ] is a schematic diagram illustrating the deployment of an application execution system architecture in a bare metal environment according to an embodiment of the present specification; [ FIG. 8B ] shows a schematic diagram of an application execution system architecture deployed in a container according to an embodiment of the present specification; [ Fig. 8C ] shows a schematic diagram of an application execution system architecture deployed in a virtual machine environment according to an embodiment of the present specification; [ FIG. 9 ] shows an example schematic diagram for deploying a new GPU node in a GPU computing cluster according to an embodiment of the present specification; [ FIG. 10 ] shows a block diagram of a computing device for executing an application program applied to a server side according to an embodiment of the present specification; and [ FIG. 11 ] A block diagram illustrating a computing device for executing an application program applied to a client according to an embodiment of the present specification.

300: GPU node

310-1, 310-2: Client

311-1, 311-2: Application Layer

312-1, 312-2: AI Framework Layer

313-1, 313-2: Application Execution Device

320: Servo side

321: Application Execution Device

322: Hardware Discovery Unit

323: Adaptation strategy creation unit

324: GPU execution resource isolation unit

325: Instruction priority management unit

326: GPU execution optimization unit

330: GPU

340: Hard drive

Claims (19)

An apparatus for executing an application program, the apparatus is applied to a server in a GPU node, and at least one GPU hardware is deployed in the GPU node, and the apparatus includes: The execution request receiving unit receives an application program interface instruction execution request from the client, the application program interface instruction execution request includes the first version information of the application program interface specified in the dynamic link library required for the execution of the application program, the first The version information is obtained in response to the client detecting that the application is activated; The adaptation processing unit, based on the application programming interface adaptation strategy, performs application programming interface adaptation processing according to the first version information and the second version information of the application programming interface, and the second version information is installed on the server. version information of the application programming interface in the driver of the at least one GPU hardware; an application execution unit that uses the adapted application programming interface to access the at least one GPU hardware to execute application programming interface instructions; and The execution result sending unit sends the execution result of the application programming interface instruction to the client. The device of claim 1, further comprising: a hardware discovery unit to discover GPU hardware in the GPU node; and The adaptation strategy creation unit creates the application programming interface adaptation strategy based on the found application programming interface compatibility list of the GPU hardware. The device of claim 1 or 2, further comprising: A GPU execution resource isolation unit that allocates isolation resources for the GPU hardware to execute the API instructions; and The instruction priority management unit manages the priority of the application program interface instruction to be executed on the GPU hardware. The device of claim 3, further comprising: The GPU execution optimization unit performs optimization processing on the GPU hardware. The apparatus according to claim 1, wherein the client and the server are located in the same device, and the communication between the client and the server is implemented by an inter-process communication mechanism. The apparatus according to claim 1, wherein the client and the server are located in different devices, and the communication between the client and the server is implemented using a network protocol. The apparatus of claim 1, wherein the client and the server are located in the same GPU node, or the client and the server are located in different GPU nodes. The device of claim 1, wherein the application execution request includes application scheduling information, the application invocation information is used to specify a target GPU hardware that needs to be accessed when the application is executed, and the The target GPU hardware is part or all of the at least one GPU hardware. A device for executing an application program, the device is applied to a client in a GPU node, wherein at least one GPU hardware is deployed in the GPU node, the device includes: a version information acquisition unit, in response to detecting that the application program is activated and executed, acquires the first version information of the application program interface specified in the dynamic link library required for the execution of the application program; An execution request sending unit sends an application programming interface instruction execution request to the server in the GPU node, where the application programming interface instruction execution request includes the first version information, so as to perform application programming interface adaptation on the server end processing and application execution processing; and The execution result receiving unit receives the execution result of the application programming interface instruction from the server, Wherein, the application program interface adaptation process is performed according to the first version information and the second version information of the application program interface based on the application program interface adaptation rule, and the application program execution process is performed using the adapted The processed API is used to access the at least one GPU hardware to execute the API instructions, and the second version information is in the driver of the at least one GPU hardware installed on the server Version information for the API. A GPU node including: A server terminal, the server terminal includes the device according to any one of request items 1 to 8; at least one client, each client comprising the apparatus of claim 9; and At least one GPU hardware. A method for executing an application program, the method is applied to a server in a GPU node, wherein at least one GPU hardware is deployed in the GPU node, the method includes: An application programming interface instruction execution request is received from the client, and the application programming interface instruction execution request includes the first version information of the application programming interface specified in the dynamic link library required for the execution of the application program, and the first version information is a response to the Acquired by the client when it detects that the application is activated and executed; Based on the application programming interface adaptation strategy, the application programming interface adaptation process is performed according to the first version information and the second version information of the application programming interface, and the second version information is the at least one version installed on the server. Version information of the API in the driver of the GPU hardware; using the adapted API to access the at least one GPU hardware to execute API instructions; and Sending the execution result of the API instruction to the client. The method of claim 11, wherein the API adaptation policy is created based on an API compatibility list of the GPU hardware. The method according to claim 11, wherein the client and the server are located in the same device, and the communication between the client and the server is implemented by using an inter-process communication mechanism. The method according to claim 11, wherein the client and the server are located in different devices, and the communication between the client and the server is implemented using a network protocol. A method for executing an application program, the method is applied to a client in a GPU node in which GPU hardware is deployed, the method comprising: In response to detecting that the application program is started to execute, obtain the first version information of the application program interface specified in the dynamic link library required for the execution of the application program; Sending an API command execution request to the server in the GPU node, the API command execution request including the first version information, so as to perform API adaptation processing and application execution on the server processing; and receiving an execution result of an API command from the server, Wherein, the application program interface adaptation process is performed according to the first version information and the second version information of the application program interface based on the application program interface adaptation rule, and the application program execution process is performed using the adapted The processed API is used to access the at least one GPU hardware to execute the API instructions, and the second version information is in the driver of the at least one GPU hardware installed on the server Version information for the API. A computing device comprising: one or more processors, and storage coupled to the one or more processors, the storage stores instructions that, when executed by the one or more processors, cause the one or more processors to perform as requested The method of any one of 11 to 14. A machine-readable storage medium storing executable instructions that, when executed, cause the machine to perform the method of any of claims 11-14. A computing device comprising: one or more processors, and storage coupled to the one or more processors, the storage stores instructions that, when executed by the one or more processors, cause the one or more processors to perform as requested 15 of the method. A machine-readable storage medium storing executable instructions that, when executed, cause the machine to perform the method of claim 15.

Images