CN119960768A - A distributed software deployment method and system for large-scale clusters - Google Patents
A distributed software deployment method and system for large-scale clusters Download PDFInfo
- Publication number
- CN119960768A CN119960768A CN202411940451.3A CN202411940451A CN119960768A CN 119960768 A CN119960768 A CN 119960768A CN 202411940451 A CN202411940451 A CN 202411940451A CN 119960768 A CN119960768 A CN 119960768A
- Authority
- CN
- China
- Prior art keywords
- task
- cluster
- deployment
- configuration
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
- G06F8/62—Uninstallation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/65—Updates
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Stored Programmes (AREA)
Abstract
本发明提供一种面向大规模集群的分布式软件部署方法及系统，该方法步骤包括：步骤1：在集群局域网中的任一节点上安装部署代理，将部署代理分别与客户端、分布式集群建立通信连接，启动部署代理扫描集群内各主机的IP地址并生成公钥；步骤2：用户通过客户端登录部署代理，检测用户是否有安装权限，如果有则创建部署任务，在分布式集群上执行部署任务；步骤3：用户通过客户端登录部署代理，检测用户是否有配置权限，如果有则进入分布式软件更新配置；步骤4：用户通过客户端登录部署代理，检测用户是否有升级权限，如果有则进入分布式软件升级配置。本发明能够智能化实现大规模集群的分布式软件部署，提升大规模集群的部署效率以及成功率。
The present invention provides a distributed software deployment method and system for large-scale clusters, the method steps include: step 1: installing a deployment agent on any node in the cluster local area network, establishing communication connections between the deployment agent and the client and the distributed cluster respectively, starting the deployment agent to scan the IP addresses of each host in the cluster and generate a public key; step 2: the user logs in to the deployment agent through the client, detects whether the user has installation authority, creates a deployment task if yes, and executes the deployment task on the distributed cluster; step 3: the user logs in to the deployment agent through the client, detects whether the user has configuration authority, and enters the distributed software update configuration if yes; step 4: the user logs in to the deployment agent through the client, detects whether the user has upgrade authority, and enters the distributed software upgrade configuration if yes. The present invention can intelligently realize the distributed software deployment of large-scale clusters, and improve the deployment efficiency and success rate of large-scale clusters.
Description
Technical Field
The invention relates to the field of distributed software systems, in particular to a distributed software deployment method and system for a large-scale cluster.
Background
The distributed software system is the software system adopting the distributed architecture, and the distributed architecture can ensure the high throughput and the high availability of the software system. In practical deployment, the distributed software system is usually installed on a plurality of computers, and the computers are communicated with each other through a network to form a distributed cluster. The size of a distributed cluster varies from a few to thousands of machines, depending on the business scenario. For large-scale distributed clusters (e.g., more than 100 nodes) with a large number of nodes, how to quickly deploy software on the distributed clusters has been a difficult problem for software operators.
Aiming at the software deployment of the distributed clusters, the prior art generally adopts manual node-by-node deployment, but the manual deployment mode is only suitable for the clusters with smaller scale, and when facing the large-scale clusters, the manual deployment mode has the disadvantages of huge workload, easy occurrence of human errors, low efficiency and difficulty in meeting the requirements of quick iteration and frequent update. In order to solve the problems, an automatic software deployment mode based on containers is adopted, for example, container arrangement tools such as an open source container arrangement platform of Kubernetes are adopted for deployment, and the tools can automatically manage the life cycle of containerized applications, including deployment, expansion, service discovery and other functions, so that the automation degree of the deployment process can be greatly improved, and human intervention is reduced. However, the deployment mode based on the container depends on a container arranging platform such as Kubernetes and the like and a hardware virtualization function, the software needs to be subjected to containerization adaptation and encapsulation in advance, the deployment mode is container mirroring, and a physical machine and a virtual machine cluster are not supported.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides the distributed software deployment and upgrading method and system which are simple in implementation method, low in cost, high in deployment intelligentization degree and efficiency, and can intelligently implement distributed software deployment of a large-scale cluster, and the deployment efficiency and success rate of the large-scale cluster are improved.
In order to solve the technical problems, the invention adopts the following technical scheme:
A distributed software deployment method facing a large-scale cluster comprises the following steps:
Installing a deployment agent on any node in a cluster local area network to be deployed, establishing communication connection between the deployment agent and a client and between the deployment agent and the distributed cluster, starting the deployment agent to automatically scan out IP addresses of all hosts in the cluster, and generating SSH public keys for all hosts in the cluster;
Step 2, the distributed software is installed and uninstalled, namely, a user logs in a deployment agent through a client, whether the user has installation permission or not, if so, a deployment task is created, deployment task parameters and task types are obtained, the configuration center transmits basic configuration data containing node roles to each node of the cluster, the configuration center checks whether the configuration data of each node in the target distributed cluster is missing or wrong, if not, the missing or required correct configuration data is obtained from the configuration center, and the deployment task is executed on each node of the distributed cluster according to the obtained deployment task parameters, task types and the configuration data, wherein the task types comprise installation and uninstallation;
Step 3, the distributed software updating configuration, namely, a user logs in a deployment agent through a client, detects whether the user has configuration permission, if so, the user enters the distributed software updating configuration, acquires configuration parameters and then sends the configuration parameters to a target distributed cluster to control the target distributed cluster to update the parameters, and performs data verification on local data of each node of the target distributed cluster and the configuration data sent by a configuration center, and if so, synchronous data is obtained according to the state of each node;
And 4, distributed software upgrading, namely, a user logs in a deployment agent through a client, detects whether the user has upgrading authority, enters distributed software upgrading configuration if the user has upgrading authority, associates a current upgrading task with a historical installation task, selects a task template corresponding to the historical installation task as a task template of the current upgrading task to acquire upgrading task parameters, uploads a software package required by upgrading, then executes upgrading on a target distributed cluster, acquires the state of each node in the target distributed cluster through a configuration center to calculate the upgrading sequence of each node, and transmits data containing the upgrading sequence to each node to update the cluster configuration.
Further, step1 includes:
Step 101, a firewall is deployed between a client and a deployment agent, and a client IP address white list is opened on a firewall wall so as to control only the client on the white list to pass through the firewall;
Step 102, after installing a deployment agent in a cluster local area network to be deployed, starting a host scanning tool for deploying agent services to automatically scan out the IP of a cluster host;
Step 103, grouping the clustered hosts IP, adding SSH public keys for each host, and generating using commands if the hosts do not have the public keys;
Step 104.6, creating a user and a password required to be used by the client on the deployment agent, setting rights, and generating a digital certificate for the client.
Further, the step 2 includes:
step 201, a user logs in a deployment agent through a client by using a pre-created user and a password;
step 202, the deployment agent checks whether the user has the installation authority, if so, the step 203 is shifted to, otherwise, the current flow is ended;
step 203, entering a deployment task management stage, and creating a deployment task;
step 204, selecting whether to import from the template, if yes, selecting a deployment template, otherwise, directly creating a task template;
Step 205, acquiring task parameters and task types according to a current task template, wherein the task parameters comprise any plurality of task labels, task remarks, task executors, script paths, script parameters and execution sequences;
Step 206, judging the current task type, if the task is an installation task, uploading a software package to be installed through a client and loading the software package to a deployment agent, turning to step 207, and if the task is an unloading task, directly turning to step 207;
Step 207, the configuration center transmits basic configuration data containing node roles to each node of the cluster;
Step 208, checking whether the configuration data of each node in the target distributed cluster is missing or wrong by the configuration center, and if the configuration data is not checked, acquiring the missing or required correct configuration data from the configuration center;
step 209, executing tasks on the distributed cluster node by node according to the acquired task parameters and configuration data until the execution of the deployment tasks on all the selected nodes is completed, wherein if the tasks are installation tasks, the software packages are installed on the distributed cluster node by node until the installation of the software packages on all the selected nodes is completed;
Step 210, checking whether the deployment tasks of all nodes in the distributed cluster are successfully executed, and if not, returning to step 207 to try to execute the tasks again.
Further, the step 3 includes:
Step 301, a user logs in a deployment agent through a client by using a pre-created user and a password;
step 302, the deployment agent checks whether the user has configuration authority, if yes, the step 303 is shifted to, otherwise, the current flow is ended;
Step 303, judging whether the configuration parameter input mode is batch import, if so, acquiring a group of configuration parameters in a batch import configuration file, otherwise, registering the configuration parameters one by one to acquire a group of configuration parameters;
Step 304, the configuration parameters obtained in the step 303 are issued to the target cluster to control the target distributed cluster to update parameters;
Step 305, performing data verification on the local data of each node of the target distributed cluster and the configuration data issued by the configuration center, if conflict data exist, updating the conflict nodes by the configuration data of the configuration center, acquiring the states of other nodes in the target distributed cluster, and calculating to obtain the latest synchronous data;
Step 306, judging whether the cluster needs to be restarted, if yes, restarting cluster nodes one by one in a rolling restarting mode, if not, exiting the current flow;
step 307, checking whether the current configuration update is successful, if not, returning to step 304 to issue the configuration parameter number again.
Further, in step 304, the configuration parameters obtained in step 303 are written into the designated file of the target cluster, or the configuration parameters obtained in step 303 are directly written into the interface of the cluster, so as to implement the issuing of the configuration parameters.
Further, step4 includes:
step 401, a user logs in a deployment agent through a client by using a user and a password which are created in advance;
Step 402, the deployment agent checks whether the user has upgrading authority, if yes, the step 403 is carried out, otherwise, the current flow is ended;
step 403, selecting a history installation task and correlating with a current upgrading task, and taking a task template of the selected history installation task as a task template of the current upgrading task to realize template multiplexing;
step 404, acquiring upgrade task parameters according to the task template obtained in step 403, wherein the upgrade task parameters comprise any one or more of upgrade scripts, upgrade configuration file paths, task responsibility people, task descriptions and rollback scripts;
Step 405, uploading a full-quantity or incremental software package to a deployment agent to execute an upgrade task, wherein the full-quantity software package is all software package data, and the incremental software package is software package data updated before and after upgrade;
Step 406, the configuration center communicates with each node in the target distributed cluster to collect the state of each node in the cluster, and calculates the upgrading sequence of each node according to the state of each node;
step 407, transmitting the data containing the upgrade sequence of each node to each node in the target distributed cluster to update the cluster configuration;
Step 408, according to the rolling upgrade mode, executing upgrade script for each node of the target distributed cluster until the upgrade of all cluster nodes is completed;
Step 409, checking whether each node in the cluster is successfully upgraded, if there is a failure in upgrading the cluster node, returning to step 406 to retry the upgrade, or performing a rollback operation to cancel the upgrade task.
A system for implementing the above-described distributed software deployment method for large-scale clusters, comprising:
the client is used for providing an interface with the deployment agent and inputting and outputting data;
the deployment agent is deployed on any node in the cluster local area network and is used for executing distributed cluster management and deploying task agents;
a distributed cluster comprising a plurality of physical machines or virtual machines;
The client and the distributed cluster are respectively communicated with the deployment agent through HTTP connection.
Further, the deployment agent comprises:
The distributed cluster management unit is used for realizing host addition, batch import, host grouping and cluster information management of the distributed clusters;
the deployment task management unit is used for realizing three types of deployment tasks, namely installation, upgrading and updating, of the distributed cluster and realizing any one or more of task editing, task execution, task rollback and task retry;
the configuration center unit is used for centrally managing the configuration parameter management function of the distributed cluster and providing corresponding configuration parameters for the distributed cluster in the process of executing the deployment task;
The software package management unit is used for realizing management of the software package;
The role and authority management unit is used for realizing user management, user authority management and user role management so as to control different users to access and operate the system according to different authorities;
The system configuration unit is used for realizing the configuration of the system, wherein the configuration comprises any one or more of starting memory size, data storage path and user black-and-white list;
and the template management unit is used for realizing the management of the task template.
Further, the distributed cluster management module further comprises a node management unit, a storage management unit and a network management unit, wherein the node management unit is used for adding and deleting nodes of the distributed cluster, the storage management unit is used for expanding and shrinking storage capacity of the distributed cluster, and the network management unit is used for managing IP network segments, gateway addresses and the like of the distributed cluster.
A computer readable storage medium storing a computer program which, when executed by a processor, implements a method as described above.
Compared with the prior art, the invention has the advantages that:
1. The invention installs the deployment agent in the cluster local area network, and uses the deployment agent to intelligently install/uninstall, update and upgrade distributed software by adopting unified processing logic, so that resource multiplexing can be realized among various processes, the logic of a user using system is simplified, the management efficiency and deployment efficiency of a large-scale cluster can be improved, the complexity of user use can be reduced, the misoperation probability of the user can be reduced, and meanwhile, a centralized configuration center is adopted to carry out data verification on installation and update and carry out cluster configuration update on the upgrading process, and the problem of configuration data deletion/error can be conveniently processed in the deployment process by effectively aiming at different role nodes, and the problem of configuration conflict can be effectively solved by synchronizing configuration data in the upgrading process.
2. According to the invention, the deployment agent is arranged in the cluster local area network, the deployment task is created through the deployment agent, and the deployment task is automatically executed on each node on the distributed cluster, so that the cluster hosts can be managed in batches, meanwhile, the initialization debugging is automatically carried out on the cluster hosts, the whole process is highly automatic, the manual bench-by-bench installation and debugging are not needed, the manual intervention process is greatly reduced, the management efficiency and the deployment efficiency of the large-scale cluster can be improved, the error risk caused by manual operation can be reduced, and the deployment success rate of the distributed software is improved.
3. The distributed software deployment process is further automatically packaged and abstracted into the reusable task template, the configuration parameters are centrally managed and automatically issued in a file form through the configuration center, the deployment parameters and the debugging software are not required to be manually modified, the distributed software is not required to be modified, and the deployment and upgrading efficiency of the distributed software can be greatly improved.
Drawings
Fig. 1 is a system architecture suitable for use in embodiments of the present invention.
FIG. 2 is a schematic diagram of a distributed software installation and uninstallation implementation in an embodiment of the present invention.
Fig. 3 is a schematic flow chart of an implementation of distributed software update configuration in an embodiment of the present invention.
Fig. 4 is a schematic diagram of a distributed software upgrade implementation flow in an embodiment of the present invention.
Fig. 5 is a schematic diagram of a system architecture according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Because the installation/uninstallation, the updating and the upgrading belong to different tasks, the traditional distributed software deployment process is usually independently executed, and the process flow differences of the installation/uninstallation, the updating and the upgrading are large, the execution flows cannot be used for resource multiplexing, for example, the installation/uninstallation does not need to consider the reserved historical data, the updating and the upgrading need to reserve the historical data, and the task flows and task scripts of the installation/uninstallation, the updating and the upgrading cannot be directly multiplexed, so that the implementation is complex, the required resources are more, and the cost is high.
For deployment of large-scale distributed software, the distributed software is logically integrated but physically divided, so that maintenance of configuration data is a difficult problem in the processes of installation, deployment, updating and upgrading, for example, configuration data matched with different roles is difficult to be issued aiming at nodes with different roles in the process of installation, the problem of missing and error of the configuration data is difficult to be processed in the process of deployment, the problem of configuration conflict is difficult to be caused by the difficulty of synchronization of the configuration data in the process of updating, and the problem of cluster node state dependence is difficult to be solved in the process of upgrading.
The invention comprehensively considers the problems, by installing the deployment agent in the cluster local area network, the deployment agent is utilized to intelligently install/uninstall, update and upgrade distributed software by adopting unified processing logic, so that resource multiplexing can be realized among all processes, the logic of a user using system is simplified, the management efficiency and the deployment efficiency of a large-scale cluster can be improved, the complexity of the user using can be reduced, the misoperation probability of the user can be reduced, meanwhile, the centralized configuration center is adopted to carry out data verification on the installation and update and carry out cluster configuration update on the upgrading process, the problem of configuration data deletion/error can be conveniently processed in the deployment process by effectively aiming at the configuration data matched with the nodes with different roles in the installation process, the problem of configuration conflict can be effectively solved by synchronizing the configuration data in the updating process, and the problem of cluster node state dependence can be solved in the upgrading process.
Fig. 1 is a system architecture suitable for the embodiment, where the system may include a client, a deployment agent and a distributed cluster, where the client is configured to provide an interface with the deployment agent to perform data input and output, the deployment agent is deployed on any node in a cluster local area network and is configured to perform distributed cluster management and deployment task agent, the distributed cluster includes multiple physical machines or virtual machines, the client and the deployment agent communicate through HTTP connection, and the deployment agent and the distributed cluster are deployed on different computers and connected through HTTP. In the initialization stage, a deployment agent is started to automatically scan out IP addresses of all hosts in a cluster, SSH public keys are generated for all hosts in the cluster, when distributed software installation and uninstallation are needed to be executed, a user logs in the deployment agent through a client, if installation permission exists, a deployment task is created, deployment task parameters and task types (installation or uninstallation) are obtained after a deployment template is selected, and then the deployment task is executed on all nodes in the distributed cluster according to the deployment task parameters.
The communication link between the client and the deployment agent in this embodiment may be reinforced by a security reinforcement manner, and specifically may be reinforced by the following manners:
1. And a firewall is deployed between the client and the deployment agent, a white list is opened on the firewall, only the clients on the white list are allowed to pass through the firewall, and the clients outside the white list are intercepted by the firewall.
2. The client must use the correct identity credentials to log into the deployment agent, which may use passwords, digital signature certificates, hardware key fobs, etc.
3. The client communicates with the deployment agent using the TLS1.3/HTTPS security protocol.
The communication link between the deployment agent and the distributed cluster may also be secured, for example, by the same security reinforcement means as the client-to-deployment agent communication link, except that no firewall is provided.
Based on the system, the distributed software deployment method for the large-scale cluster in the embodiment comprises the following steps:
Installing and initializing a deployment agent and a distributed cluster, namely installing the deployment agent on any node in a cluster local area network to be deployed, establishing communication connection between the deployment agent and a client and between the deployment agent and the distributed cluster respectively, starting the deployment agent to automatically scan out IP addresses of all hosts in the cluster, and generating SSH public keys for all hosts in the cluster.
The deployment agent can be deployed on any machine in the cluster local area network, the deployment agent and the distributed clusters can be respectively deployed on different computers, the deployment agent and the network equipment are communicated, and the client and the deployment agent can be deployed on the same computer under the condition of limited resources.
As an alternative embodiment, as shown in fig. 2, the installation and initialization of the deployment agent and the distributed cluster may take the following steps:
Step 101, a firewall is deployed between a client and a deployment agent, and a client IP address white list is opened on a firewall wall so as to control only the client on the white list to pass through the firewall;
Step 102, after installing a deployment agent in a cluster local area network to be deployed, starting a host scanning tool for deploying agent services to automatically scan out the IP of a cluster host;
Step 103, grouping the clustered hosts IP, adding SSH public keys for each host, and generating using commands if the hosts do not have the public keys;
Step 104.6, creating a user and a password required to be used by the client on the deployment agent, setting rights, and generating a digital certificate for the client.
The method comprises the steps of firstly determining the scale of a distributed cluster and the environment of the distributed cluster in the process of system installation and initialization, installing a deployment agent on any machine in the cluster local area network, then starting a host scanning tool in a distributed cluster management module of the deployment agent service, inputting network segments of the distributed cluster, automatically scanning out the IP of the cluster hosts, for example, scanning all hosts of 192.168.10.0/24 network segments, namely nmap-sn 192.168.10.0/24, grouping the IP of the cluster hosts, adding an SSH public key for each host, and if the hosts do not have the public key, using commands to generate, for example, commands, namely SSH-keygen-t rsa-b 4096, so as to complete the installation and initialization of the deployment agent and the distributed clusters.
In the process of installing the deployment agent, the method further comprises the steps of creating a user and a password which are needed to be used by the client, setting the authority and finally generating a digital certificate for the client. In the installation and initialization process of the client, the digital certificate is required to be installed on the client, and meanwhile, the IP address of the client is added to the white list on the fireproof wall, so that the installation and initialization of the client are completed. The installation and initialization of the deployment agent, client and distributed cluster are completed.
And 2, the distributed software is installed and uninstalled, namely, a user logs in a deployment agent through a client, whether the user has installation permission or not, if so, a deployment task is created, deployment task parameters and task types are acquired, basic configuration data containing node roles are issued to all nodes of the cluster by a configuration center, whether the configuration data of all the nodes in the target distributed cluster are missing or wrong is checked by the configuration center, if not, the missing or required correct configuration data is acquired from the configuration center, and the deployment task is executed on all the nodes of the distributed cluster according to the acquired deployment task parameters, task types and the configuration data, wherein the task types comprise installation and uninstallation.
As an alternative embodiment, as shown in fig. 3, the distributed software installation and removal may take the following steps:
step 201, a user logs in a deployment agent through a client by using a pre-created user and a password;
Specifically, on the client, the user logs in to the deployment agent using a pre-created user and password, if a hardware key fob is configured, the key fob needs to be inserted.
Step 202, the deployment agent checks whether the user has the installation authority (INSTALL _PRIL), if yes, the step 203 is carried out, otherwise, if the user is not granted the installation authority, the current flow is finished;
step 203, entering a deployment task management stage, and creating a deployment task;
Step 204, selecting whether to import from the template, if yes, selecting a deployment template, otherwise, directly creating a task template to create a task flow as required;
Step 205, acquiring task parameters and task types according to a current task template, wherein the task parameters comprise task labels, task remarks, task executors, script paths, script parameters, execution sequences and the like, the script paths are installation script paths if the tasks are installation tasks, the execution sequences are installation sequences, the script paths are uninstallation script paths if the tasks are uninstallation tasks, and the execution sequences are uninstallation sequences;
Step 206, judging the current task type, if the task is an installation task, uploading a software package to be installed through a client and loading the software package to a deployment agent, turning to step 207, and if the task is an unloading task, directly turning to step 207;
Step 207, the configuration center transmits basic configuration data containing node roles to each node of the cluster;
Step 208, checking whether the configuration data of each node in the target distributed cluster is missing or wrong by the configuration center, and if the configuration data is not checked, acquiring the missing or required correct configuration data from the configuration center;
step 209, executing tasks on the distributed cluster node by node according to the acquired task parameters and configuration data until the execution of the deployment tasks on all the selected nodes is completed, wherein if the tasks are installation tasks, the software packages are installed on the distributed cluster node by node until the installation of the software packages on all the selected nodes is completed;
Step 210, checking whether the deployment tasks of all nodes in the distributed cluster are successfully executed, and if not, returning to step 207 to try to execute the tasks again.
The task template is specifically a structured data structure, and includes data such as a task number, a task executor, a task authority, cluster configuration data, a task flow, a task script, a task parameter, and the like. All tasks automatically generate a task template after being successfully created, and are stored in a storage system such as a file system and a database in a lasting manner. Based on the task template, the task creation flow can be simplified, the task success rate can be improved, and task debugging and task audit can be conveniently performed.
And step 3, the distributed software updating configuration, namely, a user logs in a deployment agent through a client, detects whether the user has configuration permission, enters the distributed software updating configuration if the user has the configuration permission, acquires a group of configuration parameters through batch import or one-by-one registration of the configuration parameters, and issues the acquired configuration parameters to a target distributed cluster so as to control the target distributed cluster to update the parameters.
As an alternative embodiment, as shown in fig. 4, the distributed software update configuration may take the following steps:
Step 301, a user logs in a deployment agent through a client by using a pre-created user and a password, and if a hardware key fob is configured, the key fob needs to be inserted;
Step 302, the deployment agent checks whether the user has the configuration authority (CONFIG_PRIL), if yes, the step 303 is shifted to, otherwise, the current flow is ended;
Step 303, judging whether the configuration parameter input mode is batch import, if so, acquiring a group of configuration parameters in a batch import configuration file, otherwise, registering the configuration parameters one by one to acquire a group of configuration parameters;
Step 304, the configuration parameters obtained in the step 303 are issued to the target cluster to control the target distributed cluster to update parameters;
Step 305, performing data verification on the local data of each node of the target distributed cluster and the configuration data issued by the configuration center, if conflict data exist, updating the conflict nodes by the configuration data of the configuration center, acquiring the states of other nodes in the target distributed cluster, and calculating to obtain the latest synchronous data;
Step 306, judging whether the cluster needs to be restarted, if yes, restarting cluster nodes one by one in a rolling restarting mode, if not, exiting the current flow;
step 307, checking whether the current configuration update is successful, if not, returning to step 304 to issue the configuration parameters again.
Specifically, in step 305, a consensus algorithm may be used to calculate the synchronization data according to the states of the nodes, that is, the local data of each node of the cluster is compared with the configuration data issued by the configuration center, if there is conflict data, the local data is determined to be based on the configuration center data, and then the local data is communicated with other nodes to obtain the states of the nodes, and the latest synchronization data is calculated by the consensus algorithm to confirm that the states of the whole cluster data are consistent.
In a specific application embodiment, one profile is as follows:
cluster1.database.master=192.168.10.1
cluster1.database.slaves=192.168.10.2,192.168.10.3,192.168.10.4
cluster1.name=Cassandra_Cluster
nginx.vip=192.168.10.100
...
Alternatively, the configuration parameters may be issued in two different manners, for example, by writing the registered configuration parameters obtained in step 303 into a designated file of the target cluster, or by directly writing the registered configuration parameters obtained in step 303 into an interface (such as JMX interface) of the cluster.
In the embodiment, in the distributed software updating configuration process, a rolling restarting mode is adopted, so that the functions provided by the distributed clusters can be ensured to be uninterrupted, and the stability and the reliability of the system are improved.
And step 4, the distributed software upgrading, namely, a user logs in a deployment agent through a client, detects whether the user has upgrading authority, enters distributed software upgrading configuration if the user has upgrading authority, associates a current upgrading task with a historical installation task, selects a task template corresponding to the historical installation task as a task template of the current upgrading task to obtain upgrading task parameters, uploads a software package required by upgrading, and then executes upgrading on the target distributed cluster.
As shown in fig. 4, the steps include:
Step 401, a user logs in a deployment agent through a client by using a pre-created user and a password, and if a hardware key fob is configured, the key fob needs to be inserted;
Step 402, the deployment agent checks whether the user has upgrade authority (UPDATE_PRIL), if yes, go to step 403, otherwise finish the current flow;
Step 403, selecting a history installation task and correlating with the current upgrading task, taking a task template of the selected history installation task as a task template of the current upgrading task to realize template multiplexing, namely directly multiplexing the history task template;
step 404, acquiring upgrade task parameters according to the task template obtained in step 403, wherein the upgrade task parameters comprise upgrade scripts, upgrade configuration file paths, task responsibility people, task descriptions, rollback scripts and the like;
step 405, uploading a full-quantity or incremental software package to a deployment agent, wherein the full-quantity software package is all software package data, the incremental software package is updated before and after upgrading, and the data transmission quantity can be reduced by uploading the incremental software package;
Step 405, uploading a full-quantity or incremental software package to a deployment agent to execute an upgrade task, wherein the full-quantity software package is all software package data, and the incremental software package is software package data updated before and after upgrade;
Step 406, the configuration center communicates with each node in the target distributed cluster to collect the state of each node in the cluster, and calculates the upgrading sequence of each node according to the state of each node;
step 407, transmitting the data containing the upgrade sequence of each node to each node in the target distributed cluster to update the cluster configuration;
Step 408, according to the rolling upgrade mode, executing upgrade script for each node of the target distributed cluster until the upgrade of all cluster nodes is completed;
Step 409, checking whether each node in the cluster is successfully upgraded, if there is a failure in upgrading the cluster node, returning to step 406 to retry the upgrade, or performing a rollback operation to cancel the upgrade task.
In step 403, when the history installation task is associated with the current upgrade task, the following two association methods may be adopted:
1. and selecting and associating, namely providing all the history installation tasks listed by the system for the user to select, and associating the history installation tasks selected by the user with the current upgrading task after receiving the selection of the user.
2. And automatically associating, namely selecting the latest installation task from the historical tasks by the system according to the information such as the user roles, task names and the like to associate with the current upgrading task.
The two modes can be switched, and the user selects the association mode.
After the association is successful, the current upgrading task multiplexes the data in the task template of the history installation task, for example, the data comprise cluster configuration, task parameters, task scripts and the like, so that the current upgrading task can be rapidly executed by taking the data of the history installation task.
It should be understood that the present embodiment is not limited to the order of the steps, and the steps of installing/uninstalling, updating and upgrading may be performed according to actual requirements, for example, the steps of installing/uninstalling, updating and upgrading may be performed first and then, or the steps of installing/uninstalling, updating and upgrading may be performed periodically or dynamically according to actual requirements.
According to the embodiment, the upgrading task is executed according to the rolling upgrading mode in the upgrading process of the distributed software, the functions of the distributed clusters in the upgrading process can be guaranteed to be uninterrupted, the stable reliability of the system is improved, and the upgrading process can be simplified and the upgrading efficiency is further improved by adopting a mode of multiplexing historical task templates.
According to the method, the distributed software deployment flow is automatically packaged and abstracted into the reusable task template, the configuration parameters are centrally managed and automatically issued in a file form through the configuration center, the deployment parameters and the debugging software are not required to be manually modified, the distributed software is not required to be transformed, and the deployment and upgrading efficiency of the distributed software can be greatly improved.
Further, in the deployment process, all deployment tasks including operators, operation time, software version, parameter files, log files and the like can be automatically recorded, so that user audit and disk duplication are facilitated, and the success rate of distributed software deployment can be further improved.
In a specific application embodiment, firstly, a software and hardware system is installed and debugged according to the system composition shown in fig. 1, after a software package and a configuration file are prepared and initialization configuration is executed, a deployment task is executed according to the process according to the installation/uninstallation, parameter update or upgrade task to be executed, whether the software is successfully deployed in a distributed cluster is confirmed, and if not, the software is operated according to a retry or rollback process.
In a specific application embodiment, as shown in fig. 5, a graphical interface is provided by a client as an interface of man-machine interaction, and a data input/output function, for example, a software package import/export function, is provided by an IO device, and is communicated with a deployment agent through a network device, where the client and the deployment agent can be deployed on the same computer under the condition of limited resources. The distributed cluster is composed of a plurality of physical machines or virtual machines, and can provide functions of node management, storage management and network management, wherein nodes of the distributed cluster are added and deleted through a node management unit, the storage capacity of the distributed cluster is expanded and reduced through a storage management unit, and IP network segments, gateway addresses and the like of the distributed cluster are managed through a network management unit.
In a specific application embodiment, as shown in fig. 5, the deployment agent may include:
The distributed cluster management unit is used for realizing host addition, batch import, host grouping and cluster information management of the distributed clusters;
the deployment task management unit is used for realizing three types of deployment tasks, namely installation, upgrading and updating, of the distributed cluster and realizing any one or more of task editing, task execution, task rollback and task retry;
the configuration center unit is used for centrally managing the configuration parameter management function of the distributed cluster and providing corresponding configuration parameters for the distributed cluster in the process of executing the deployment task;
The software package management unit is used for realizing management of the software package;
The role and authority management unit is used for realizing user management, user authority management and user role management so as to control different users to access and operate the system according to different authorities;
The system configuration unit is used for realizing the configuration of the system, wherein the configuration comprises any one or more of starting memory size, data storage path and user black-and-white list;
A template management unit for realizing the management of the task template
By the structure, functions of host manual addition, batch import, host grouping, cluster information management and the like can be provided for the distributed clusters through the distributed cluster management unit, the deployment task management unit can provide three types of deployment tasks for the distributed clusters and provide functions of task editing, task execution, task rollback, task retry and the like, the configuration center unit can provide a centralized configuration parameter management function for the distributed clusters, the distributed clusters can acquire corresponding configuration parameters from the configuration center when the deployment tasks are executed, the software package management unit can provide functions of import and export of software packages, version management, signature verification and the like, the role and authority management unit can provide functions of user management, user authority management and user role management, different users can conveniently access and operate the system according to different authorities, the system configuration unit can provide configuration functions of the system, such as functions of starting memory size, data storage path, user black and white, the template management unit can provide functions of storing, editing, importing and exporting task templates, and rapidly deploying a task list are convenient.
Optionally, the deployment agent may also configure the remote terminal to provide the user with the functionality to remotely connect to the distributed cluster hosts through which the user may directly operate the hosts. The deployment agent may also configure the operation and maintenance help unit for providing user assistance, fault handling, use guidance, etc. functions, facilitating a user's more friendly use of the system.
The present embodiment further provides a computer readable storage medium storing a computer program which, when executed by a processor, implements a method as described above.
It will be appreciated by those skilled in the art that the above-described embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411940451.3A CN119960768A (en) | 2024-12-26 | 2024-12-26 | A distributed software deployment method and system for large-scale clusters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411940451.3A CN119960768A (en) | 2024-12-26 | 2024-12-26 | A distributed software deployment method and system for large-scale clusters |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN119960768A true CN119960768A (en) | 2025-05-09 |
Family
ID=95588641
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411940451.3A Pending CN119960768A (en) | 2024-12-26 | 2024-12-26 | A distributed software deployment method and system for large-scale clusters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119960768A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190369980A1 (en) * | 2018-06-04 | 2019-12-05 | Palantir Technologies Inc. | Constraint-based upgrade and deployment |
| CN111131146A (en) * | 2019-11-08 | 2020-05-08 | 北京航空航天大学 | Multi-supercomputing center software system deployment and incremental updating method in wide area environment |
| CN116962260A (en) * | 2022-04-15 | 2023-10-27 | 腾讯科技（深圳）有限公司 | Cluster security inspection method, device, equipment and storage medium |
| CN117850812A (en) * | 2024-01-15 | 2024-04-09 | 江苏电力信息技术有限公司 | Distributed application dynamic deployment and upgrading method based on agent |
-
2024
- 2024-12-26 CN CN202411940451.3A patent/CN119960768A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190369980A1 (en) * | 2018-06-04 | 2019-12-05 | Palantir Technologies Inc. | Constraint-based upgrade and deployment |
| CN111131146A (en) * | 2019-11-08 | 2020-05-08 | 北京航空航天大学 | Multi-supercomputing center software system deployment and incremental updating method in wide area environment |
| CN116962260A (en) * | 2022-04-15 | 2023-10-27 | 腾讯科技（深圳）有限公司 | Cluster security inspection method, device, equipment and storage medium |
| CN117850812A (en) * | 2024-01-15 | 2024-04-09 | 江苏电力信息技术有限公司 | Distributed application dynamic deployment and upgrading method based on agent |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110572436B (en) | Multi-place cross-cluster server deployment method and system | |
| CN107566165B (en) | Method and system for discovering and deploying available resources of power cloud data center | |
| US9137111B2 (en) | Discovering, validating, and configuring hardware-inventory components | |
| KR101954480B1 (en) | Automated build-out of a cloud-computing stamp | |
| CN110752947A (en) | K8s cluster deployment method and device, and deployment platform | |
| CN107870772B (en) | Cross-environment application deployment method, platform, system and readable storage medium | |
| US8230416B2 (en) | System, method and article of manufacture for using shadow installations of software modules during distributed system upgrade | |
| CN110286921B (en) | CDH method for automatically installing distributed big data platform | |
| CN112631614A (en) | Application deployment method and device, computer equipment and storage medium | |
| CN109474443B (en) | Configuration method, device and system of newly added server and communication equipment | |
| CN110347609B (en) | Method and device for testing blockchain software | |
| CN112434008A (en) | Distributed database upgrading method, device and medium | |
| CN109799998B (en) | OpenStack cluster configuration and batch deployment method and system | |
| CN117908904B (en) | K8S cluster deployment and operation and maintenance management method and system | |
| EP4130982B1 (en) | Network-based solution module deployment platform | |
| CN113238864A (en) | Python-based MongoDB cluster deployment method and system | |
| CN105404530B (en) | It is a kind of to realize easy deployment and the system and method using private clound | |
| CN113330419B (en) | Equipment application installation method, device, equipment and storage medium | |
| CN110795105A (en) | Automatic compiling and deploying method and system for power distribution automation master station system | |
| CN119512568A (en) | A convenient deployment method for multiple optional components based on DevOps | |
| CN106301877A (en) | The upgrade method of a kind of Virtual NE and device | |
| CN119960768A (en) | A distributed software deployment method and system for large-scale clusters | |
| CN117692323A (en) | Method for realizing deployment of big data clusters by hybrid architecture based on secure | |
| CN113886008A (en) | Virtual machine management method, system, device and medium | |
| CN106161107A (en) | Multimachine intelligent service deployment system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |