Detailed Description
The embodiment of the application provides a method, a device, equipment and a medium for cross-cloud resource operation based on task arrangement, which not only can flexibly combine and schedule multi-cloud resources in a task arrangement mode, but also can ensure traceability of all resource operations through a unified log tracking system and can rapidly locate and solve problems when the problems occur.
The technical scheme in the embodiment of the application has the following overall thought:
1. task orchestration driven cloud resource operation flow
The invention abstracts the operation in the business service into task scheduling logic and manages the task scheduling logic through the scheduling service. The task orchestration logic includes sequential definitions of main tasks, sub-tasks, and actions of the tasks. When the business service needs to perform resource operation, the scheduling service sequentially calls service interfaces of all cloud providers according to predefined task scheduling logic. Decoupling with different cloud manufacturers is achieved through an adapter mode, and it is ensured that resource operations of the different cloud manufacturers can be executed according to a unified flow.
2. Unified cloud resource operation log tracking
The invention provides a unified log tracking system, which records the execution condition of each task, including the context information of the task and the manufacturer call log. In the log system, not only the execution state of each main task can be checked, but also an alarm can be set according to the log level, and when the system fails to operate, alarm mechanisms such as nails are automatically triggered, so that the operation and maintenance efficiency and the system reliability are improved.
The implementation flow is as follows:
1. Task orchestration logic is defined in the scheduling service:
Defining task orchestration logic in a system, comprising:
a main task, such as an application agency;
Subtasks, wherein each subtask corresponds to specific steps of the main task, such as IP creation, instance creation, network card creation, IP binding, deployment agent and the like;
defining the execution sequence of each subtask to ensure that all necessary operations are completed in sequence;
2. service invocation scheduling service
The business service specifies the provider and the primary task to be performed and invokes the dispatch service. For example, select to perform an agent application task for alicloud.
3. Scheduling service reception processing
And the scheduling service receives the newly added scheduling task request, and extracts and stores the request parameters into the Redis according to the configuration of the newly built main task.
3.1, Distributing the main task, namely, in the main task management process, the dispatching service inquires the main task in the initial state and generates and adds initial subtasks according to the defined task relation;
3.2 subtask_item in the execution of the subtask, the dispatch service first checks if the main task is in error, and if so, cancels the execution of the following subtasks. If the concurrency threshold is reached, execution of the subtasks is delayed. Obtaining the supplier adaptation service and action to be sent, request parameters, asynchronously sending requests, and updating the subtask state into a waiting callback according to the subtask configuration;
4. vendor adapted service reception process
4.1, Firstly, obtaining a manufacturer and an action to be executed according to a requested parameter, wherein the parameter is a parameter set in 3.2, finding an adapter, wherein the adapter is used for realizing interface call of each provider, completing actual action, obtaining the parameter from redis, executing action call of the provider, and sequentially requesting interfaces of external cloud manufacturers according to specific logic in the action call, wherein the logic is an execution step of specific actions of each subtask;
4.2, recording calling information of an external cloud manufacturer by the system through a buried point, generating a sub operation log (calling method, state, starting time, ending time, execution times, parameters, return value and whether the operation is abnormal) and recording an operation result;
5. scheduling service receive callback processing
In the callback processing of the dispatch service manager, callback information is received, and concurrent locking is carried out according to the main task ID at the moment so as to ensure the concurrency safety of tasks, and ensure that only one sub task is allowed to carry out callback processing at the same moment in the running process of the sub task under the same main task. If the locking operation fails, a message is sent that is delayed by 10 seconds to re-attempt the task at a later time. If successful, the result is updated to the Redis cache, and necessary parameter entering information is provided for the subsequent tasks. When completing one subtask, callback processing needs to judge whether to execute the execution of the next subtask;
Then judging the execution condition of the subtasks:
5.1 if the subtask fails and the retry times are reached, marking the error of the main task, updating the state of the subtask into an error, and finally uploading a main operation log (comprising a link ID, an error reason (which subtask is abnormal and causes failure), abnormal information and occurrence time) to finish the flow;
5.2 if the subtask fails but the retry times are not reached, updating the state of the subtask to be in execution, and then the system re-executes the step 3.2, namely, the subtask is called again and the attempt is continued;
5.3 when the subtask is successful, the system checks if all pre-tasks of the next step have been successfully executed. If the pre-task is not completed, the sub-task state is updated to be successful and is directly returned, if the pre-task is completed, the system adds a next sub-task in an initial state and continues to execute the step 3.2;
5.4 if no subsequent subtasks need to be executed, marking the main task as completed, updating the final state of the main task as successful, and completing the whole flow;
6. Through the integral operation log, the problem source can be tracked rapidly, and efficient fault investigation is realized.
Example 1
As shown in fig. 1, the present embodiment provides a method for operating across cloud resources based on task orchestration, including the following steps:
step 1, abstracting a received scheduling task request into task scheduling logic, and calling a service interface of a corresponding cloud provider to execute all operations according to the task scheduling logic, wherein the task scheduling logic comprises a main task, a subtask and a sequence of subtask actions;
and step 2, recording all execution conditions and recording the execution conditions into a set log file.
In this embodiment, preferably, the step 1 specifically includes:
Step 11, abstracting a scheduling task request into task scheduling logic according to the received scheduling task request, wherein the task scheduling logic comprises a main task, a subtask and a sequence of subtask actions;
Step 12, generating and adding initial subtasks according to the main task, wherein each subtask is provided with corresponding task configuration, and the task configuration comprises the steps of sending supplier adaptation service, actions and request parameters to be sent, sending a request mode, and updating the subtask state into a waiting callback;
Step 13, corresponding subtasks are sequentially executed according to the execution sequence of the subtasks, actions corresponding to the tasks are executed according to task configuration, action calling is carried out through the adapter of the cloud manufacturer according to request parameters, a return result is obtained, and calling information of the cloud manufacturer is recorded in a sub operation log;
And 14, if the current sub-task is successfully executed, executing the next sub-task until all the sub-tasks are executed, recording the execution condition into a main task operation log, if the current sub-task is failed to be executed, ending the execution, updating the sub-task state into an error, marking the main task error, and recording the execution condition into the main task operation log.
In this embodiment, preferably, the step 14 specifically includes allowing only one sub-task to perform callback processing at the same time when the main task is executed, performing locking operation according to the main task ID if the current sub-task is executed successfully, delaying the first time when the locking operation fails, performing callback processing again, performing callback processing if the locking is successful, performing unlocking operation after the callback processing is completed, and performing execution of the next sub-task;
And if the current subtask fails to be executed, re-executing the subtask after the set time, ending the execution only after the same subtask fails to be set for times, updating the state of the subtask into an error, marking the error of the main task, and recording the execution condition in the main task operation log.
In this embodiment, preferably, the step 2 specifically includes recording all execution conditions including context information of a task and vendor call, recording the execution conditions into a set log file, where the log file includes a main operation log and a sub operation log, the main operation log includes a link ID, an error cause, exception information and occurrence time, the sub operation log includes a called method, a state, a start time, an end time, execution times, parameters, a return value and whether the operation of the main task or the sub task is abnormal, and triggering an alarm mechanism according to the setting when the operation of the main task or the sub task fails.
Based on the same inventive concept, the application also provides a device corresponding to the method in the first embodiment, and the details of the second embodiment are shown.
Example two
As shown in fig. 2, in this embodiment, an apparatus for cross-cloud resource operation based on task orchestration is provided, including the following modules:
The execution operation module is used for abstracting the received scheduling task request into task scheduling logic, and calling a service interface of a corresponding cloud provider to execute all operations according to the task scheduling logic, wherein the task scheduling logic comprises a main task, a subtask and the sequence of the subtask actions;
And the log recording module records all execution conditions and records the execution conditions into a set log file.
In this embodiment, preferably, the execution operation module specifically includes:
the task scheduling unit abstracts the scheduling task request into task scheduling logic according to the received scheduling task request, wherein the task scheduling logic comprises a main task, a subtask and a sequence of subtask actions;
the subtask setting unit is used for generating and adding initial subtasks according to the main task, and setting corresponding task configuration for each subtask, wherein the task configuration comprises a supplier adaptation service, an action and a request parameter which need to be sent, a request mode is sent, and the subtask state is updated to be a waiting callback;
The execution recording unit sequentially executes corresponding subtasks according to the execution sequence of the subtasks, obtains actions corresponding to the execution of the tasks according to task configuration, performs action calling through the adapter of the cloud manufacturer according to request parameters, obtains a return result, and records calling information of the cloud manufacturer into a sub operation log;
And the final execution unit is used for executing the next subtask until all the subtasks are executed and recording the execution condition into the main task operation log if the current subtask is executed successfully, ending the execution if the current subtask is executed successfully, updating the subtask state into an error, marking the main task error and recording the execution condition into the main task operation log.
In this embodiment, preferably, the final execution unit specifically allows only one sub-task to perform callback processing at the same time when the main task is executed, if the current sub-task is executed successfully, locking operation is performed according to the main task ID, if the locking operation fails, the first time is delayed, callback processing is performed again, and if the locking is successful, callback processing is performed;
And if the current subtask fails to be executed, re-executing the subtask after the set time, ending the execution only after the same subtask fails to be set for times, updating the state of the subtask into an error, marking the error of the main task, and recording the execution condition in the main task operation log.
In this embodiment, the log recording module preferably records all execution conditions including context information of a task and vendor call, and records the execution conditions into a set log file, where the log file includes a main operation log and a sub operation log, the main operation log includes a link ID, an error cause, exception information and occurrence time, the sub operation log includes a calling method, a state, a start time, an end time, execution times, parameters, a return value and whether the operation of the main task or the sub task fails, and triggers an alarm mechanism according to the setting.
Since the device described in the second embodiment of the present invention is a device for implementing the method described in the first embodiment of the present invention, based on the method described in the first embodiment of the present invention, a person skilled in the art can understand the specific structure and the deformation of the device, and thus the detailed description thereof is omitted herein. All devices used in the method according to the first embodiment of the present invention are within the scope of the present invention.
Based on the same inventive concept, the application provides an electronic device embodiment corresponding to the first embodiment, and the details of the third embodiment are shown in the specification.
Example III
The present embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where any implementation of the first embodiment may be implemented when the processor executes the computer program.
Since the electronic device described in this embodiment is a device for implementing the method in the first embodiment of the present application, those skilled in the art will be able to understand the specific implementation of the electronic device and various modifications thereof based on the method described in the first embodiment of the present application, so how the electronic device implements the method in the embodiment of the present application will not be described in detail herein. The apparatus used to implement the methods of embodiments of the present application will be within the scope of the intended protection of the present application.
Based on the same inventive concept, the application provides a storage medium corresponding to the first embodiment, and the detail of the fourth embodiment is shown in the specification.
Example IV
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, can implement any of the implementation modes of the embodiment.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that the specific embodiments described are illustrative only and not intended to limit the scope of the invention, and that equivalent modifications and variations of the invention in light of the spirit of the invention will be covered by the claims of the present invention.