WO2025026280A1 - Model deployment method and system, and electronic device and computer-readable storage medium - Google Patents

Abstract

The present disclosure relates to large model technology and the field of model deployment. Disclosed are a model deployment method and system, and an electronic device and a computer-readable storage medium. The method comprises: acquiring a model file of a model to be deployed; distributing the model file to a plurality of storage devices, wherein different storage devices are geographically deployed in different regions; and mounting the storage devices to a target server cluster, which is deployed in the same region as the storage devices, among a plurality of server clusters, and thus the model to be deployed is deployed to the plurality of server clusters. The present disclosure solves the technical problem in the related art of the efficiency of cross-region deployment of a model being relatively low.

Description

Model deployment method, system, electronic device, and computer-readable storage medium Technical Field

The present disclosure relates to the field of large model technology and model deployment, and in particular, to a model deployment method, system, electronic device, and computer-readable storage medium.

Background Art

The current large models store a large number of parameters and calculation graph structures, which makes the model files very large. It takes a long time to load the model when starting the service. In addition, due to factors such as region, network, and hardware, the loading time will be further extended when cross-region file transfer is involved, resulting in low model deployment efficiency.

To address the above-mentioned problems, no effective solution has been proposed yet.

Summary of the invention

The embodiments of the present disclosure provide a model deployment method, system, electronic device, and computer-readable storage medium to at least solve the technical problem of low efficiency in cross-regional deployment of models in related technologies.

According to one aspect of an embodiment of the present disclosure, a model deployment method is provided, including: obtaining a model file of a model to be deployed; distributing the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; mounting the storage device on a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters.

According to another aspect of an embodiment of the present disclosure, a model deployment method is also provided, including: in response to receiving a model file of a model to be deployed, storing the model file in a central warehouse; in response to receiving a model distribution request, distributing the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions, and different server clusters are also deployed in different regions; in response to receiving a server deployment request, based on the server deployment request, mounting the storage device to a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters.

According to another aspect of an embodiment of the present disclosure, a model deployment method is also provided, including: obtaining a model file of a model to be deployed by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the model file; distributing the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; when deploying an inference service of the model to be deployed, mounting the storage device on a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters to obtain a deployment result of the model to be deployed; outputting the deployment result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the deployment result.

According to another aspect of the embodiment of the present disclosure, a model deployment system is also provided, including: a plurality of server clusters, where different server clusters are geographically deployed in different regions; a plurality of storage devices, where different storage devices are geographically deployed in different regions; a control device, connected to the storage device and the server cluster, for Distribute the model files of the model to be deployed to the storage device, and when deploying the inference service of the model to be deployed, mount the storage device to the target server cluster deployed in the same region as the storage device, so that the model to be deployed can be deployed to multiple server clusters.

According to another aspect of the embodiments of the present disclosure, there is further provided an electronic device, including: a memory storing an executable program; and a processor for running the program, wherein the program executes any one of the methods in the above embodiments when it is run.

According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the methods in the above embodiments.

In the embodiments of the present disclosure, a model file of a model to be deployed can be obtained; the model file can be distributed to multiple storage devices, wherein different storage devices are geographically deployed in different regions; the storage device can be mounted on a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed can be deployed to multiple server clusters, thereby improving the processing efficiency of the model in different regions; it is easy to notice that the model file can be distributed to storage devices deployed in different regions, so as to utilize the network between the storage device and the server cluster in the same region to achieve high-speed interconnection between the storage device and the server cluster, eliminate the transmission of files across regions, thereby shortening the loading time of the model file, thereby improving the processing efficiency of the model in different regions, and further solving the technical problem of low efficiency of cross-regional deployment of models in related technologies.

It is easily noted that the above general description and the following detailed description are merely for exemplifying and explaining the present disclosure, and do not constitute a limitation of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

FIG1 is a schematic diagram of a hardware environment of a virtual reality device according to a model deployment method according to an embodiment of the present disclosure;

FIG2 is a flow chart of a model deployment method according to Embodiment 1 of the present disclosure;

FIG3 is a structural diagram of a model deployment method according to an embodiment of the present disclosure;

FIG4 is a flow chart of another model deployment method according to an embodiment of the present disclosure;

FIG5 is a flow chart of another model deployment method according to an embodiment of the present disclosure;

FIG6 is a flow chart of a model deployment method according to Embodiment 2 of the present disclosure;

FIG7 is a flow chart of a model deployment method according to Embodiment 3 of the present disclosure;

FIG8 is a schematic diagram of a model deployment device according to Embodiment 4 of the present disclosure;

FIG9 is a schematic diagram of a model deployment device according to Embodiment 5 of the present disclosure;

FIG10 is a schematic diagram of a model deployment device according to Embodiment 6 of the present disclosure;

FIG11 is a schematic diagram of a model deployment system according to Embodiment 7 of the present disclosure;

FIG. 12 is a structural block diagram of a computer terminal according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

The technical solution provided by the present disclosure is mainly implemented by large-scale model technology. The large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions or even more than ten trillion model parameters. The large model can also be called the foundation model/foundation model (Foundation Model). The large model is pre-trained by large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization ability, such as large-scale language model (Large Language Model, LLM), multi-modal pre-training model (multi-modal pre-training model), etc.

It should be noted that when the large model is actually applied, the pre-trained model can be fine-tuned through a small number of samples so that the large model can be applied to different tasks. For example, the large model can be widely used in natural language processing (NLP), computer vision and other fields, and can be specifically applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, etc. It can also be widely used in natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiment of the present disclosure, the deployment of the large model in the model deployment scenario is used as an example for explanation.

First, some nouns or terms that appear in the process of describing the embodiments of the present disclosure are subject to the following explanations:

Network Attached Storage (NAS): provides file sharing services over the network and has other additional functions, which can provide a convenient and reliable way to store and share data;

Cloud Object Storage Service (OSS): It can provide cloud storage services. Used to store and manage massive amounts of unstructured data;

Virtual Private Cloud (VPC): A virtual network environment created in a cloud computing environment that provides an isolated and secure way to deploy and manage cloud resources.

Online service platform (Elastic Algorithm Service, referred to as EAS): It can be a platform for deploying and managing machine learning models. EAS provides users with a convenient way to deploy, run and manage their own machine learning algorithms and models, and provides scalable computing resources and high-performance computing capabilities;

Container orchestration platform (Kubernetes, abbreviated as K8S): can organize and manage containerized applications, helping users simplify application deployment, management, and expansion.

In order to solve the above problems, the present disclosure proposes a method based on multi-regional storage of storage devices, which deploys storage devices and services in the same virtual private network in the same region to achieve high-speed interconnection, eliminate cross-regional file transfer, thereby shortening the loading time of model files and further improving the elasticity of services.

Example 1

According to an embodiment of the present disclosure, a model deployment method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

Considering the huge amount of model parameters of the large model and the limited computing resources of the mobile terminal, the above-mentioned model deployment method provided in the embodiment of the present disclosure can be applied to the application scenario shown in Figure 1, but is not limited thereto. Figure 1 is a schematic diagram of the hardware environment of a virtual reality device according to a model deployment method of an embodiment of the present disclosure. In the application scenario shown in Figure 1, the large model is deployed in a server 10, and the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of networks. The client device 20 here may include but is not limited to: a smart phone, a tablet computer, a laptop computer, a PDA, a personal computer, a smart home device, a car-mounted device, etc. The client device 20 can interact with the user through a graphical user interface to implement the call to the large model, thereby implementing the method provided in the embodiment of the present disclosure.

In the embodiments of the present disclosure, the system composed of the client device and the server can perform the following steps: the client device generates a model file of the model to be deployed; the server obtains the model file of the model to be deployed and multiple server clusters; distributes the model file to multiple storage devices; mounts the storage device to the target server cluster deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters. It should be noted that the embodiments of the present disclosure can be performed in the client device when the operating resources of the client device can meet the deployment and operation conditions of the large model.

In the above operating environment, the present disclosure provides a model deployment method as shown in Figure 2. Figure 2 is a flow chart of the model deployment method according to Embodiment 1 of the present disclosure. As shown in Figure 2, the method may include the following steps:

Step S202, obtaining the model file of the model to be deployed.

The above-mentioned model to be deployed can be any model to be deployed. The model to be deployed can be a large model, a neural network Model, language processing model, image processing model, where a large model can refer to a machine learning or deep learning model with a large number of parameters and powerful computing power. In the embodiments of the present disclosure, the model to be deployed is a large model as an example for explanation, and the model to be deployed can be determined according to actual needs.

The above-mentioned model file may be a model file of a trained model to be deployed, wherein the model file includes but is not limited to the parameters and structure of the model. This is only an example, and the specific model file may be defined according to actual usage.

In an optional embodiment, a model file of a model to be deployed may be obtained, and a plurality of server clusters may be determined, wherein different server clusters are geographically deployed in different regions.

The above-mentioned multiple server clusters can be used to provide services for users. The above-mentioned different regions can be different countries, different provinces, different cities, etc., and the different regions are not specifically limited here.

In an optional embodiment, after the training of the model to be deployed is completed and the model file is generated, the model file can be stored in a cloud server. When the model to be deployed is deployed, the model file can be retrieved from the cloud server and distributed to storage devices in multiple different regions.

Step S204: distribute the model file to multiple storage devices.

Among them, different storage devices are deployed in different regions geographically.

The above storage device may be a hardware storage device, for example, a NAS, but is not limited thereto.

In an optional embodiment, the model file can be distributed to multiple storage devices according to the distribution request initiated by the user. Optionally, the model management and control platform can retrieve the model file from the cloud server according to the distribution request initiated by the user, and distribute and store the model file to multiple storage devices.

Step S206, mounting the storage device to a target server cluster in the multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to the multiple server clusters.

The above-mentioned mounting refers to connecting the storage device to the server of the target server cluster so that the model files in the storage device can be visible and accessible on the servers of the target server cluster.

In an optional embodiment, the storage device can be mounted to a target server cluster deployed in the same region as the storage device, so that users can quickly access the model file through the mounted storage device on the target server cluster, thereby more efficiently using the services provided by the model.

Through the above steps, the model file of the model to be deployed can be obtained; the model file is distributed to multiple storage devices, wherein different storage devices are geographically deployed in different regions; the storage device is mounted on a target server cluster in multiple server clusters and deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters, thereby improving the processing efficiency of the model in different regions; it is easy to notice that the model file can be distributed to storage devices deployed in different regions, so as to utilize the network between the storage device and the server cluster in the same region to achieve high-speed interconnection between the storage device and the server cluster, eliminate cross-regional file transmission, thereby shortening the loading time of the model file, thereby improving the processing efficiency of the model in different regions, and further solving the technical problem of low efficiency of cross-regional deployment of models in related technologies.

In the above embodiment of the present disclosure, the storage device includes: a network attached storage, wherein the model file is distributed to The plurality of storage devices comprises: sending the model file to the plurality of network attached storages through a public network and storing the model file in the plurality of network attached storages.

The above-mentioned network attached storage can be a storage device that provides file sharing services through the network and has other additional functions. Among them, the network attached storage can also provide users with a convenient and reliable way to store and share data.

The above-mentioned public network may be an open, shared and widely covered network.

In an optional embodiment, since the public network has a wide coverage area, the model file can be sent to multiple network attached storages through the public network so that the multiple network attached storages can receive the model file. After receiving the model file, the multiple network attached storages can store the model file.

In another optional embodiment, in order to improve the security during the storage of the model file, the model file can be encrypted before sending the model file to multiple network attached storages through a public network to obtain an encrypted model file, and then the encrypted model file can be sent to the multiple network attached storages through the public network. After receiving the encrypted model file, the multiple network attached storages can decrypt the encrypted model file to obtain the model file and store the model file.

In the above embodiment of the present disclosure, when deploying the inference service of the model to be deployed, the storage device is mounted to a target server cluster in multiple server clusters that is deployed in the same region as the storage device, including: determining the target virtual private network corresponding to the storage device based on the target region where the storage device is deployed; obtaining the server cluster corresponding to the target virtual private network to obtain the target server cluster, wherein different server clusters correspond to different virtual private networks; when deploying the inference service of the model to be deployed, the storage device is mounted to a server in the target server cluster.

The above-mentioned target region can be determined according to the geographical location where the storage device is deployed, or according to the country, province, etc. where the storage device is deployed. The method for determining the target region is not limited here, and the method for determining the target region for storage device deployment can be set according to actual conditions.

The target virtual private network (Virtual Private Cloud, referred to as VPC) can provide an isolated and secure way to deploy and manage resources, and can also increase the transmission speed between files through the virtual private network. Optionally, the target virtual private network can be a virtual network environment created in a cloud computing environment.

In an optional embodiment, the target virtual private network corresponding to the storage device can be determined based on the target region where the storage device is deployed, the server cluster connected to the target virtual private network can be obtained, and one or more server clusters can be selected from the server clusters connected to the target virtual private network as the target server cluster; optionally, the server cluster that is geographically closest to the storage device can be selected from the server clusters connected to the target virtual private network as the above-mentioned target server cluster. The method of determining the target server cluster here is only for example description and is not specifically limited. It can be set according to actual conditions.

In another optional embodiment, after obtaining the target region for storage device deployment, the server cluster in the target region may be searched according to the target virtual private network, and one or more of the server clusters in the target region may be searched. Multiple server clusters are used as the target server cluster. Optionally, the server cluster closest to the storage device can be used as the target server cluster. The specific method for determining the target server cluster is not limited here.

In another optional embodiment, when deploying the inference service of the model to be deployed, the storage device can be connected to the server in the target server cluster so that the model file in the storage device can be read on the server through the intranet.

In the above embodiment of the present disclosure, after mounting the storage device to a target server cluster in multiple server clusters and deploying the storage device in the same region, the method also includes: building an elastic scheduling cluster of the target server cluster; based on the elastic scheduling cluster, determining the computing resources required to deploy the model to be deployed.

The above-mentioned elastic scheduling cluster is a cluster management method based on cloud computing and virtualization technology, which can automatically adjust the computing resources in the cluster according to changes in the system load to meet different needs. The elastic scheduling cluster can adjust the cluster size according to the load situation, including adding or reducing computing nodes to cope with high or low load situations. This cluster management method can improve the flexibility and availability of the system, while also saving resources and costs.

The above-mentioned elastic scaling capability can automatically adjust the resources and capacity of the target service cluster according to real-time demand and load conditions to meet user needs; the elastic scaling capability can automatically increase or decrease the number of server instances in the target server cluster according to different situations to achieve load balancing and high availability.

In an optional embodiment, an elastic scheduling cluster can be built based on an online service platform (Elastic Algorithm Service, referred to as EAS) or a container orchestration platform (Kubernetes, referred to as K8S). The elastic scheduling cluster can provide elastic scaling capabilities for the target service cluster so that the target service cluster can automatically adjust system resources and capacity to meet user needs based on actual demand and load conditions.

In another optional embodiment, when the load of the target server cluster is low, the number of servers can be reduced to save costs, and when the load of the target server cluster is high, the number of servers can be increased to improve performance and capacity; optionally, the number of servers in the target server cluster can be managed according to pre-set rules, and the number of servers in the target server cluster can also be dynamically adjusted based on real-time monitoring and analysis.

In another optional embodiment, after building an elastic scheduling cluster of the target server cluster, it is necessary to determine the computing resources required to deploy the model to be deployed, such as memory, storage and other resources, and configure and allocate them according to the computing requirements of the model. By determining the computing resources required for deployment, it can be ensured that the model to be deployed can run normally on the elastic scheduling cluster and meet the required performance and availability requirements. In addition, sufficient computing resources can be provided by building an elastic scheduling cluster to support the deployment and operation of the model to be deployed.

In the above embodiment of the present disclosure, the method also includes: distributing preset resources to multiple server clusters; distributing model files to multiple storage devices, including: distributing model files to multiple storage devices when the preset resources are distributed.

The above-mentioned preset resources can be various resources shared and allocated in the server cluster, wherein the preset resources can be computing resources, network resources, storage resources, software resources, load balancing resources, and the examples are described here only. The specific preset resources can be set according to the actual situation.

The above-mentioned preset resources may be pre-set resource types and resource quantities, and the preset resources are not limited here.

The above-mentioned preset resources can also determine the preset resources according to the computing resources required for the deployment of the model to be deployed, and distribute the preset resources to multiple server clusters, so as to facilitate the subsequent deployment of the model to be deployed through multiple server clusters.

In an optional embodiment, the model management and control platform can be used to provide preset resources, including but not limited to managing cluster metadata, service deployment, automatic/scheduled scalers, monitoring data, model distribution, monitoring resource usage, and grayscale/rollback.

In another optional embodiment, the preset resources can be distributed to multiple server clusters, and after the preset resources are distributed, the model file can be distributed to multiple storage devices, so that the multiple storage devices can subsequently use the preset resources of the target server in the server cluster to deploy the model to be deployed according to the model file.

In the above embodiment of the present disclosure, obtaining the model file of the model to be deployed includes: obtaining the model file of the model to be deployed from a central warehouse, wherein the model file is pre-uploaded to the central warehouse.

The central warehouse mentioned above can be a cloud object storage (Object Storage Service, referred to as OSS), but is not limited to this, and is only used as an example. Among them, the central warehouse can be used to provide cloud storage services and can be used to store and manage massive unstructured data.

The above-mentioned central warehouse can communicate with the server cluster and transfer files with the storage devices on the server cluster. According to the user's model deployment instructions, the central warehouse can automatically distribute the model files of the model to be deployed to multiple storage devices mounted on multiple server clusters.

In an optional embodiment, since the central warehouse has a large memory, after the training of the model to be deployed is completed, the model file of the model to be deployed can be pre-uploaded to the central warehouse. When the model to be deployed needs to be deployed, the model file corresponding to the model to be deployed can be retrieved from the central warehouse according to the model deployment instruction, and the model file can be distributed to multiple storage devices.

In the above embodiments of the present disclosure, the model to be deployed is a large model.

The aforementioned large model may be a model with a large storage capacity and a strong computing capability.

FIG3 is a structural diagram of a model deployment method according to an embodiment of the present disclosure. As shown in FIG3 , the model management and control platform can store the model files of the model to be deployed in the central warehouse, wherein the model management and control platform can be used to provide preset resources, for example, to manage cluster metadata, service deployment, auto/cron scaler, monitoring data, model distribute, monitor resource usage, gray/rollback, and manage multiple accounts. The preset resources can be distributed to multiple server clusters. When the preset resources are distributed, the model files can be distributed to multiple storage devices. As shown in FIG3 , it can include region A, region B, and region C, wherein region A includes server cluster A, server cluster B, and network attached storage, region B includes server cluster A, server cluster B, and network attached storage, and region C includes server cluster A, server cluster B, and network attached storage. It includes server cluster A, server cluster B, and network attached storage. The network attached storage in different regions can be mounted on the servers in the server cluster, so that the agents in the server cluster can store resources for model files in the network attached storage, and the servers in the server cluster can load resources for model files in the network attached storage. An elastic scheduling cluster can be built based on the online service platform or container orchestration platform, and elastic scaling capabilities can be provided for the target service cluster based on the elastic scheduling cluster, so that the target service cluster can automatically adjust system resources and capacity to meet user needs based on actual needs and load conditions.

The present disclosure may include two model deployment scenarios, wherein the first model deployment scenario may be a scenario where models, programs, data, and images are deployed separately. FIG4 is a flowchart of another model deployment method according to an embodiment of the present disclosure. As shown in FIG4, the method includes:

Step S401, distributing preset resources to multiple server clusters;

The model management and control platform in Figure 3 can be used to provide the above-mentioned preset resources.

Step S402, when the preset resources are distributed, the model file is distributed to multiple storage devices;

Optionally, if the preset resources are not distributed completely, if the step of distributing the model file to multiple storage devices is executed, an error prompt is returned until the preset resources are distributed completely, and then the step of distributing the model file to multiple storage devices is executed.

Step S403, determining a target virtual private network corresponding to the storage device based on the target region where the storage device is deployed;

Step S404, obtaining a server cluster corresponding to the target virtual private network, and obtaining a target server cluster;

Step S405, when deploying the inference service of the model to be deployed, the storage device is mounted to the server in the target server cluster.

The second model deployment scenario may be a mirror deployment scenario. In this process, there is no need to execute the resource distribution process, and the service deployment can be performed directly. FIG5 is a flow chart of another model deployment method according to an embodiment of the present disclosure. As shown in FIG5, the method includes:

Step S501, obtaining a model file of a model to be deployed;

Step S502, distributing the model file to multiple storage devices;

Step S503, determining a target virtual private network corresponding to the storage device based on the target region where the storage device is deployed;

Step S504, obtaining a server cluster corresponding to the target virtual private network, and obtaining a target server cluster;

Step S505, when deploying the inference service of the model to be deployed, the storage device is mounted to the server in the target server cluster.

The following are the experimental results of a single-machine test. This disclosure mainly uses NAS mount model files. The OSS mount model file and the NAS mount model file can be compared using a 13B parameter GPT model (25GB size):

1. Use the dd command to test, time dd if = /home/workspace/model/pytorch_model.bin of = /dev/null bs = 100K count = 100000000;

a. Use OSS to mount the model file. 25716611305 bytes (26GB, 24GiB) have been copied. 115.811s, 222MB/s;

b. Use NAS to mount the model file. 25716611305 bytes (26GB, 24GiB) have been copied. 4.23957s, 6.1GB/s;

2. Large model inference service startup speed test

a. Use OSS to mount the model file. The total time taken to start the service is: 5m28.259s;

b. Use NAS to mount the model file. The total time taken to start the service is: 4m2.845s;

By comparison, it can be seen that the startup time of NAS storage is shorter than that of OSS.

The present disclosure aims to provide a system for improving the multi-regional and large-scale elasticity of large model services. The system is based on NAS storage to achieve the distribution of model files in multiple regions, and automatically mounts the NAS in the region to load the model files when deploying the inference service. Since the NAS and the inference service are in the same region VPC, high-speed interconnection can be achieved, thereby shortening the model loading time, speeding up the service startup speed, and improving elasticity.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present disclosure.

Example 2

According to an embodiment of the present disclosure, a model deployment method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

FIG. 6 is a flow chart of a model deployment method according to Embodiment 2 of the present disclosure. As shown in FIG. 6 , the method The steps include:

Step S602, in response to receiving the model file of the model to be deployed, storing the model file in the central warehouse;

Step S604, in response to receiving the model distribution request, distributing the model file to multiple storage devices;

Among them, different storage devices are deployed in different regions geographically, and different server clusters are also deployed in different regions.

The above-mentioned model distribution request may be a request for distributing the model to a storage device in advance when the model to be deployed is deployed remotely. Optionally, the above-mentioned model distribution request may be generated by touching the interactive interface. The specific method for generating the above-mentioned model distribution request is not limited here.

Step S606, in response to receiving the server deployment request, based on the server deployment request, the storage device is mounted to a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters.

The above-mentioned server deployment request may be a request generated when a user needs to deploy a model to be deployed. Optionally, the above-mentioned server deployment request may be generated by touching an interactive interface. The specific method of generating the above-mentioned server deployment request is not limited here.

Through the above steps, in response to receiving the model file of the model to be deployed, the model file is stored in the central warehouse; in response to receiving the model distribution request, the model file is distributed to multiple storage devices, wherein different storage devices are geographically deployed in different regions, and different server clusters are also deployed in different regions; in response to receiving the server deployment request, based on the server deployment request, the storage device is mounted to the target server cluster deployed in the same region as the storage device in the multiple server clusters, so that the model to be deployed is deployed to multiple server clusters, thereby improving the processing efficiency of the model in different regions; it is easy to notice that the model file can be distributed to the storage devices deployed in different regions, so as to utilize the network between the storage device and the server cluster in the same region to realize the high-speed interconnection between the storage device and the server cluster, eliminate the transmission of cross-regional files, thereby shortening the loading time of the model file, thereby improving the processing efficiency of the model in different regions, and further solving the technical problem of low efficiency of cross-regional deployment of models in related technologies.

It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

Example 3

According to an embodiment of the present disclosure, a model deployment method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

FIG. 7 is a flow chart of a model deployment method according to Embodiment 3 of the present disclosure. As shown in FIG. 7 , the method includes the following steps:

Step S702: Acquire the model file of the model to be deployed by calling the first interface.

The first interface includes a first parameter, and a parameter value of the first parameter is a model file.

The first interface in the above steps can be an interface for data interaction between the cloud server and the client. The client can pass the model file of the model to be deployed into the interface function as the first parameter of the interface function to achieve the purpose of uploading the model file of the model to be deployed to the cloud server.

Step S704: distribute the model file to multiple storage devices.

Among them, different storage devices are deployed in different regions geographically.

Step S706, when deploying the inference service of the model to be deployed, the storage device is mounted to the target server cluster in the multiple server clusters and deployed in the same region as the storage device, so that the model to be deployed is deployed to the multiple server clusters to obtain the deployment result of the model to be deployed.

Step S708: output the deployment result by calling the second interface.

The second interface includes a second parameter, and a parameter value of the second parameter is a deployment result.

The above-mentioned second interface can be an interface for data interaction between the cloud server and the client. The cloud server can pass the deployment result to the interface function as the second parameter of the interface function to achieve the purpose of sending the deployment result to the client.

Through the above steps, the model file of the model to be deployed is obtained by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the model file; the model file is distributed to multiple storage devices, wherein different storage devices are geographically deployed in different regions; when deploying the inference service of the model to be deployed, the storage device is mounted to a target server cluster in the same region as the storage device in multiple server clusters, so that the model to be deployed is deployed to multiple server clusters, and the deployment result of the model to be deployed is obtained; the deployment result is output by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the deployment result, thereby improving the processing efficiency of the model in different regions; it is easy to notice that the model file can be distributed to the storage devices deployed in different regions, so as to utilize the network between the storage device and the server cluster in the same region to realize high-speed interconnection between the storage device and the server cluster, eliminate the transmission of cross-regional files, thereby shortening the loading time of the model file, thereby improving the processing efficiency of the model in different regions, and thus solving the technical problem of low efficiency of cross-regional deployment of models in related technologies.

It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

Example 4

According to an embodiment of the present disclosure, a model deployment device for implementing the above-mentioned model deployment method is also provided. Figure 8 is a schematic diagram of a model deployment device according to embodiment 4 of the present disclosure. As shown in Figure 8, the device 800 includes: an acquisition module 802, a distribution module 804, and a mounting module 806.

The acquisition module is configured to acquire the model file of the model to be deployed; the distribution module is configured to distribute the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; the mounting module is configured to mount the storage device to multiple server clusters deployed in the same region as the storage device. On the target server cluster, the model to be deployed can be deployed to multiple server clusters.

It should be noted that the acquisition module 802, the distribution module 804, and the mounting module 806 correspond to steps S202 to S206 in Example 1, and the three modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above-mentioned modules may also be part of the device and may run in the server 10 provided in Example 1.

In the above embodiments of the present disclosure, the storage device includes: a network attached storage, wherein the distribution module is further used to send the model file to multiple network attached storages through a public network and store the model file in the multiple network attached storages.

In the above-mentioned embodiment of the present disclosure, the mounting module is also used to determine the target virtual private network corresponding to the storage device based on the target region where the storage device is deployed, obtain the server cluster corresponding to the target virtual private network, and obtain the target server cluster, wherein different server clusters correspond to different virtual private networks, and when deploying the inference service of the model to be deployed, the storage device is mounted to the server in the target server cluster.

In the above-mentioned embodiment of the present disclosure, the mounting module is also used to determine the target virtual private network corresponding to the storage device based on the target region where the storage device is deployed, obtain the server cluster corresponding to the target virtual private network, and obtain the target server cluster, wherein different server clusters correspond to different virtual private networks.

In the above embodiments of the present disclosure, the device further includes: a building module and a providing module.

Among them, the construction module is configured to construct an elastic scheduling cluster; the provision module is configured to provide elastic scaling capabilities for the target server cluster based on the elastic scheduling cluster.

In the above embodiments of the present disclosure, the distribution module is further used to distribute the preset resources to multiple server clusters; the distribution module is further used to distribute the model file to multiple storage devices when the preset resources are distributed.

In the above embodiment of the present disclosure, the acquisition module is further used to acquire the model file of the model to be deployed from the central warehouse, wherein the model file is uploaded to the central warehouse in advance.

In the above embodiments of the present disclosure, the model to be deployed is a large model.

It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

Example 5

According to an embodiment of the present disclosure, a model deployment device for implementing the above-mentioned model deployment method is also provided. Figure 9 is a schematic diagram of a model deployment device according to embodiment 5 of the present disclosure. As shown in Figure 9, the device 900 includes: a storage module 902, a distribution module 904, and a mounting module 906.

The storage module is configured to store the model file of the model to be deployed in the central warehouse in response to receiving the model file; the distribution module is configured to distribute the model file stored in the central warehouse to multiple storage devices in response to receiving the model distribution request, wherein different storage devices are geographically deployed in different regions, and different server clusters are also deployed in different regions; the mounting module is configured to mount the storage device to multiple server clusters deployed in the same region as the storage device based on the server deployment request in response to receiving the server deployment request. The target server clusters in the region can deploy the model to multiple server clusters.

It should be noted that the storage module 902, distribution module 904, and mounting module 906 correspond to steps S602 to S606 in Example 2, and the three modules and corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Example 1. It should be noted that the modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the modules may also be part of a device and run in the server 10 provided in Example 1.

It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

Example 6

According to an embodiment of the present disclosure, a model deployment device for implementing the above-mentioned model deployment method is also provided. Figure 10 is a schematic diagram of a model deployment device according to embodiment 6 of the present disclosure. As shown in Figure 10, the device 1000 includes: an acquisition module 1002, a distribution module 1004, a mounting module 1006, and an output module 1008.

Among them, the acquisition module is configured to obtain the model file of the model to be deployed by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the model file; the distribution module is configured to distribute the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; the mounting module is configured to mount the storage device to a target server cluster deployed in the same region as the storage device in multiple server clusters when deploying the inference service of the model to be deployed, so that the model to be deployed is deployed to multiple server clusters to obtain the deployment result of the model to be deployed; the output module is configured to output the deployment result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the deployment result.

It should be noted that the acquisition module 1002, distribution module 1004, mounting module 1006, and output module 1008 correspond to steps S702 to S708 in Example 3, and the four modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above-mentioned modules may also be part of the device and may be run in the server 10 provided in Example 1.

It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

Example 7

According to an embodiment of the present disclosure, a model deployment system for implementing the above-mentioned model deployment method is also provided. FIG. 11 is a schematic diagram of a model deployment system according to Embodiment 7 of the present disclosure. As shown in FIG. 11 , the system 1100 includes:

A plurality of server clusters 1102, where different server clusters are geographically deployed in different regions;

A plurality of storage devices 1104, where different storage devices are geographically deployed in different regions;

The control device 1106 is connected to the storage device and the server cluster, and is used to distribute the model files of the model to be deployed to the storage device, and mount the storage device to the target server cluster deployed in the same region as the storage device. To deploy the model to be deployed to multiple server clusters.

In the above embodiments of the present disclosure, the storage device 1104 includes: a network attached storage, which is connected to the control device through a public network.

In the above embodiments of the present disclosure, the target server cluster is connected to the storage device via a target virtual private network.

In the above embodiment of the present disclosure, the system further includes: an elastic scheduling cluster, which is used to provide elastic scaling capabilities for the target server cluster.

It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

Example 8

The embodiment of the present disclosure may provide an electronic device, which may be any electronic device in a group of electronic devices. Optionally, in this embodiment, the electronic device may also be replaced by a terminal device such as a mobile terminal.

Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the model deployment method: obtain the model file of the model to be deployed, and multiple server clusters, wherein different server clusters are geographically deployed in different regions; distribute the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; mount the storage device to the target server cluster deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters.

Optionally, Figure 12 is a block diagram of a computer terminal according to an embodiment of the present disclosure. As shown in Figure 12, the computer terminal A may include: one or more (only one is shown in the figure) processors 102, a memory 104, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

Among them, the memory can be used to store software programs and modules, such as the program instructions/modules corresponding to the model deployment method and device in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the above-mentioned model deployment method is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: obtain the model file of the model to be deployed; distribute the model file to multiple storage devices, where different storage devices are geographically deployed in different regions; mount the storage device to a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters.

Optionally, the processor may also execute the following steps: Sent to and stored in a plurality of network attached storages.

Optionally, the processor may also execute program codes of the following steps: based on the target region where the storage device is deployed, determining a target server cluster deployed in the target region; and mounting the storage device to a server in the target server cluster.

Optionally, the processor may also execute the program code of the following steps: based on the target region where the storage device is deployed, determine the target virtual private network corresponding to the storage device; obtain the server cluster corresponding to the target virtual private network to obtain the target server cluster, wherein different server clusters correspond to different virtual private networks; and when deploying the inference service of the model to be deployed, mount the storage device to the server in the target server cluster.

Optionally, the processor may also execute program codes of the following steps: constructing an elastic scheduling cluster; and providing elastic scaling capabilities for the target server cluster based on the elastic scheduling cluster.

Optionally, the processor may also execute program codes of the following steps: distributing preset resources to multiple server clusters; and distributing the model file to multiple storage devices when the preset resources have been distributed.

Optionally, the processor may also execute the program code of the following steps: obtaining a model file of the model to be deployed from the central warehouse, wherein the model file is uploaded to the central warehouse in advance.

The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: in response to receiving the model file of the model to be deployed, the model file is stored in the central warehouse; in response to receiving the model distribution request, the model file is distributed to multiple storage devices, wherein different storage devices are geographically deployed in different regions, and different server clusters are also deployed in different regions; in response to receiving the server deployment request, based on the server deployment request, the storage device is mounted on the target server cluster deployed in the same region as the storage device in the multiple server clusters, so that the model to be deployed is deployed to multiple server clusters.

The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: obtain the model file of the model to be deployed by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the model file; distribute the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; when deploying the inference service of the model to be deployed, mount the storage device to a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters to obtain the deployment result of the model to be deployed; output the deployment result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the deployment result.

The disclosed embodiment provides a model deployment method, including: obtaining a model file of a model to be deployed; distributing the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; mounting the storage device to multiple server clusters and a target server cluster deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters, thereby improving the processing efficiency of the model in different regions; it is easy to notice that the model file can be distributed to storage devices deployed in different regions, so as to utilize the network between the storage device and the server cluster in the same region to realize high-speed interconnection between the storage device and the server cluster, eliminate the transmission of cross-regional files, thereby shortening the loading time of the model file, thereby improving The high model processing efficiency in different regions solves the technical problem of low efficiency of cross-regional deployment of models in related technologies.

It will be understood by those skilled in the art that the structure shown in FIG. 12 is for illustration only, and the computer terminal may also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, and a mobile Internet device (Mobile Internet Devices, MID), PAD, etc. FIG. 12 does not limit the structure of the above-mentioned electronic device. For example, the computer terminal A may also include more or fewer components (such as a network interface, a display device, etc.) than those shown in FIG. 12, or may have a configuration different from that shown in FIG. 12.

A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

Example 6

The embodiment of the present disclosure also provides a computer-readable storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the model deployment method provided in the first embodiment.

Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining a model file of a model to be deployed, wherein different server clusters are geographically deployed in different regions; distributing the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; mounting the storage device on a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters.

Optionally, the storage medium is further configured to store program codes for executing the following steps: sending the model file to a plurality of network attached storages via a public network, and storing the model file in the plurality of network attached storages.

Optionally, the storage medium is also configured to store program codes for executing the following steps: determining a target virtual private network corresponding to the storage device based on a target region for storage device deployment; obtaining a server cluster corresponding to the target virtual private network to obtain a target server cluster, wherein different server clusters correspond to different virtual private networks; and mounting the storage device on a server in the target server cluster when deploying an inference service for the model to be deployed.

Optionally, the storage medium is further configured to store program codes for executing the following steps: constructing an elastic scheduling cluster; and providing elastic scaling capabilities for the target server cluster based on the elastic scheduling cluster.

Optionally, the storage medium is further configured to store program codes for executing the following steps: distributing preset resources to multiple server clusters; and distributing the model file to multiple storage devices when the preset resources have been distributed.

Optionally, the storage medium is further configured to store program codes for executing the following steps: The model file of the model to be deployed is obtained from the library, where the model file is uploaded to the central warehouse in advance.

Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: in response to receiving a model file of a model to be deployed, storing the model file in a central warehouse; in response to receiving a model distribution request, distributing the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions, and different server clusters are also deployed in different regions; in response to receiving a server deployment request, based on the server deployment request, mounting the storage device to a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters.

Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: obtaining a model file of the model to be deployed by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the model file; distributing the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; when deploying the inference service of the model to be deployed, mounting the storage device on a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters to obtain a deployment result of the model to be deployed; outputting the deployment result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the deployment result.

By adopting the embodiment of the present disclosure, a model deployment method is provided, including: obtaining a model file of a model to be deployed; distributing the model file to multiple storage devices, wherein different storage devices are geographically deployed in different regions; mounting the storage device to a target server cluster in multiple server clusters and deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters, thereby improving the processing efficiency of the model in different regions; it is easy to notice that the model file can be distributed to storage devices deployed in different regions, so as to utilize the network between the storage device and the server cluster in the same region to realize high-speed interconnection between the storage device and the server cluster, eliminate the transmission of cross-regional files, thereby shortening the loading time of the model file, thereby improving the processing efficiency of the model in different regions, and further solving the technical problem of low efficiency of cross-regional deployment of models in related technologies.

The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to On multiple network units, some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present disclosure. These improvements and modifications should also be regarded as the scope of protection of the present disclosure.

Claims (20)

A model deployment method, comprising: Get the model file of the model to be deployed; Distributing the model file to a plurality of storage devices, wherein different storage devices are geographically deployed in different regions; When deploying the inference service of the model to be deployed, the storage device is mounted on a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to the multiple server clusters. The method according to claim 1, wherein the storage device comprises: a network attached storage, and distributing the model file to a plurality of storage devices comprises: The model file is sent to the plurality of network attached storages through a public network and stored in the plurality of network attached storages. The method according to claim 1, wherein, when deploying the inference service of the model to be deployed, mounting the storage device to a target server cluster in a plurality of server clusters that is deployed in the same region as the storage device comprises: Determining a target virtual private network corresponding to the storage device based on the target region where the storage device is deployed; Acquire a server cluster corresponding to the target virtual private network to obtain the target server cluster, wherein different server clusters correspond to different virtual private networks; When deploying the inference service of the model to be deployed, the storage device is mounted to a server in the target server cluster. The method according to claim 1, wherein, after mounting the storage device to a target server cluster in a plurality of server clusters and deployed in the same region as the storage device, the method further comprises: Building an elastic scheduling cluster of the target server cluster; Based on the elastic scheduling cluster, computing resources required to deploy the to-be-deployed model are determined. The method according to claim 1, wherein the method further comprises: Distributing the preset resources to the plurality of server clusters; The model file is distributed to multiple storage devices. The method according to claim 5, wherein distributing the model file to multiple storage devices comprises: When the preset resources are distributed, the model file is distributed to the plurality of storage devices. The method according to claim 1, wherein obtaining a model file of the model to be deployed comprises: The model file of the model to be deployed is obtained from the central warehouse, wherein the model file is pre-uploaded to the central warehouse. The method according to claim 7, wherein the central warehouse can be a cloud object storage. The method according to claim 1, wherein the model to be deployed is a large language model. A model deployment method, comprising: In response to receiving a model file of a model to be deployed, storing the model file in a central repository; In response to receiving a model distribution request, distributing the model file stored in the central warehouse to a plurality of storage devices, wherein different storage devices are geographically deployed in different regions, and different server clusters are also deployed in the different regions; In response to receiving a server deployment request, based on the server deployment request, the storage device is mounted to a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to the multiple server clusters. The method according to claim 10, wherein the storage device comprises: a network attached storage, and distributing the model file stored in the central warehouse to a plurality of storage devices comprises: The model file stored in the central warehouse is sent to the plurality of network attached storages through a public network and stored in the plurality of network attached storages. A model deployment method, comprising: Acquire a model file of the model to be deployed by calling a first interface, wherein the first interface includes a first parameter, and a parameter value of the first parameter is the model file; Distributing the model file to a plurality of storage devices, wherein different storage devices are geographically deployed in different regions; When deploying the inference service of the model to be deployed, the storage device is mounted on a target server cluster in multiple server clusters that is deployed in the same region as the storage device, so that the model to be deployed is deployed to the multiple server clusters, and a deployment result of the model to be deployed is obtained; The deployment result is output by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the deployment result. The method according to claim 12, wherein the storage device comprises: a network attached storage, and distributing the model file to a plurality of storage devices comprises: The model file is sent to the plurality of network attached storages through a public network and stored in the plurality of network attached storages. A model deployment system, comprising: A plurality of server clusters, wherein different server clusters are geographically deployed in different regions; A plurality of storage devices, different storage devices being geographically deployed in different regions; A control device connected to the plurality of storage devices and the plurality of server clusters for storing The model file of the model is distributed to multiple storage devices, and when deploying the inference service of the model to be deployed, the storage device is mounted to a target server cluster deployed in the same region as the storage device, so that the model to be deployed is deployed to multiple server clusters. The system of claim 14, wherein the storage device comprises: The network attached storage is connected to the control device via a public network. The system according to claim 14, wherein the server cluster and storage device in the same region are connected via a virtual private network. The system according to claim 14, wherein the target server cluster is connected to the storage device via a target virtual private network. The system according to claim 14, wherein the system further comprises: The elastic scheduling cluster is used to provide elastic scaling capabilities for the target server cluster. An electronic device, comprising: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 13 when running. A computer-readable storage medium, wherein the computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the method described in any one of claims 1 to 13.