WO2021019281A1

WO2021019281A1 - Service level agreement (sla) prediction of feature composition method

Info

Publication number: WO2021019281A1
Application number: PCT/IB2019/056504
Authority: WO
Inventors: Catherine Truchan; Chunyan Fu
Original assignee: Telefonaktiebolaget LM Ericsson AB
Current assignee: Telefonaktiebolaget LM Ericsson AB
Priority date: 2019-07-30
Filing date: 2019-07-30
Publication date: 2021-02-04
Anticipated expiration: 2022-01-30
Also published as: EP4004820A1

Abstract

A method to validate Feature Composition (FC). According to one aspect, the method includes utilizing an offline Feature Composition Generator (FCG) that is utilizing a genetic algorithm (GA) to create an offline list of valid Feature Compositions (FCs), testing offline the list of valid FCs to determine offline SLA conformance results for each of the FCs of the list of valid FCs, providing the offline SLA conformance results to an offline SLA predictor trainer (SPT) to be used as training data, training the SPT using the training data to create an SLA prediction model, providing the SLA prediction model to one or more online SLA predictors to determine an online SLA prediction result for a real-time requested FC, and providing the online SLA conformance result to the offline FCG, wherein the FCG uses the online SLA conformance result in the updating of the offline list of Feature Compositions (FCs).

Description

SERVICE LEVEL AGREEMENT (SLA) PREDICTION OF FEATURE COMPOSITION METHOD

TECHNICAL FIELD

This disclosure relates to predicting the SLA for microservice-based applications, and in particular to predicting the SLA of the feature composition of microservice-based applications.

BACKGROUND

In the microservices paradigm/architecture, applications are composed of smaller independent services that implement different features. For instance, web services are composed of individual operations such as deposit and withdrawal operations in banking. In telecommunications, these features may include call forwarding, conference calling and call display. There is a need to validate the interaction of these features/microservices, which is particularly challenging due to the high frequency of microservice releases. Features are generally added incrementally to an application and when several features are added, there may be resulting interactions that severely damage the system. In particular, the interaction of features may exacerbate the application performance.

One difficulty with the interaction of features (microservices) is that such interaction may negatively impact the application performance. There is a lack of performance engineering approaches that take into account the particularities of microservices. For instance, the Continuous Delivery (CD) of microservices poses challenges due to the dynamic nature of the releases and the most important stage of CD/CI (Continuous Integration) is the validation of the developed software.

Genetic algorithm (GA) is an evolutionary algorithm that is inspired by Charles Darwin’s theory of natural selection where the fittest individuals are selected for reproduction in order to produce offspring of the next generation. A pseudo code example of GA can be found in Table 1. GA is often used to solve optimization problems and due to their robustness and efficiency GA is used for maximizing the cloud resource sharing and minimizing the task scheduling time.

Table 1 - GA Pseudo Code GA has also been used for optimizing the performance of service composition in a cloud environment. For example, GA has been used for optimizing both service placement on virtual machines (VMs) and service composition graphs in order to maximize the overall quality of service (QoS) of composed service. SUMMARY

In some aspects, a method is disclosed to predict the SLA of applications composed of several microservices using Reinforcement Learning (RL)-based Genetic Algorithm (GA). In this aspect, the method includes the utilization of an SLA Prediction Trainer (SPT) that generates a prediction model for the SLA conformance of microservices-based Feature Composition (FC). This method improves SLA prediction by combining the offline SLA Conformance Testing (CT) results with the online SLA Conformance Results. The method defines a set of valid Feature Compositions (FCs) using GA selection, crossover and mutation operations augmented by composition policies, maintains a repository of valid FCs, executes offline CTs for all valid FCs, aggregates the FC testing results and sends these results to the SLA Prediction Trainer (SPT), generates a prediction model which updates the Online SLA Predictor (OSP) accordingly. In some aspects, the method includes utilizing an online microservice-based SLA Predictor that predicts SLA conformation of microservices-based Feature Composition (FC). The method improves SLA prediction via input from SPT, feeds the SPT with FC’s offline SLA conformance results and actual resources required which are used for training the SPT to achieve better prediction accuracy, and the method also obtains online input (actual resources and Service Level Indicator (SLI)) from a system monitor.

According to another aspect, a method to validate Feature Composition (FC) can be implemented by creating an offline list of valid Feature Compositions (FCs), testing offline the list of valid FCs to determine offline SLA conformance results for each of the FCs of the list of valid FCs, providing the offline SLA conformance results to an offline SLA predictor trainer (SPT) to be used as training data, training the SPT using the training data to create an SLA prediction model, providing the SLA prediction model to one or more online SLA predictors to determine an online SLA prediction result for a real-time requested FC and providing the online SLA conformance result to the offline Feature Composition Generator (FCG), wherein the FCG uses the online SLA conformance result in the updating of the offline list of Feature Compositions (FCs).

In some of these aspects, the offline SLA conformance results can be defined as an offline list of SLA compliant FCs and an offline list of SLA non-compliant FCs. The creation of the offline list of valid FCs may include inputting a list of FCs into an FCG. The FCG may utilize a GA to create a list of FCs and applying relevant composition policies to the list of FCs to create the list of valid FCs. The utilization of the GA by the FCG may include performing the GA operations of selection, crossover and/or mutation on the list of FCs. In some of these aspects, each of the FCs in the offline list of valid FCs is defined as including a feature set, associated configurations, required resources and service level indicators.

In some aspects, the testing offline of the list of valid FC includes inputting the list of valid FCs into a Composition Test Manager (CTM), providing, by the CTM, the list of valid FCs to one or more offline Composition Testers (CT) microservices and testing each of the FCs of the list of valid FCs by the one or more CTs to determine offline SLA conformance results for each of the FCs. In some aspect, the method to validate Feature Composition (FC) can be implemented by further including the providing, by the CTM, the offline SLA conformance results for each of the FCs to the FCG and updating, by the FCG, the list of valid FCs based on the offline SLA conformance results. These aspects may also include training the SPT using a machine learning algorithm using the training data to create a prediction model. The machine learning algorithm may be a Recurrent Neural Network (RNN) algorithm.

Still in further aspects, the method to validate Feature Composition (FC) can be implemented by further including allowing or prohibiting the real-time requested FC based on the online SLA conformance prediction provided by the SLA Predictor and the current available resources provided by the System Monitor.

In additional aspects, the online SLA conformance prediction is tested by the Feature Composition Execution. And based on the execution, the System SLA Analytics will provide the online SLA conformance result that may include an SLA fulfillment status for the real-time requested FC, the actual SLIs and the actual resources used by the real-time requested FC. In these aspects, the FCG may perform the GA operation of mutation on the list of FCs to include actual service level indicators and actual resources during the updating of the list of FCs.

Further, the FCG may perform the GA operation of mutation once the difference between the required resources and service level indicators with the actual resources and actual service level indicators meets an established threshold.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present embodiments, and the attendant advantages and features thereof, will be more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings wherein: FIG. 1 is a schematic diagram of an embodiment of a Feature Composition SLA prediction system, according to various exemplary embodiments of the present disclosure.

FIG. 2 illustrates an exemplary Feature Composition, according to various exemplary embodiments of the present disclosure.

FIG. 3 illustrates an exemplary Feature Composition one-point crossover yielding valid offspring, according to various exemplary embodiments of the present disclosure.

FIG. 4 illustrates an exemplary Feature Composition two-point crossover yielding invalid offspring, according to various exemplary embodiments of the present disclosure.

FIG. 5 illustrates an exemplary Feature Composition mutation, according to various exemplary embodiments of the present disclosure.

FIG. 6 illustrates an exemplary process of offline Feature Composition SLA prediction, according to various exemplary embodiments of the present disclosure.

FIG. 7 illustrates an exemplary process of offline Feature Composition mutation, according to various exemplary embodiments of the present disclosure.

DETAILED DESCRIPTION

Before describing in detail exemplary embodiments, it is noted that the embodiments reside primarily in combinations of apparatus components and processing steps related to a real-time SLA predictor to validate the Feature Compositions (FCs). Accordingly, components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

As used herein, relational terms, such as“first” and“second,”“top” and “bottom,” and the like, may be used solely to distinguish one entity or element from another entity or element without necessarily requiring or implying any physical or logical relationship or order between such entities or elements. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the concepts described herein. As used herein, the singular forms“a”,“an” and“the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,”“comprising,”“includes” and/or“including” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

In embodiments described herein, the joining term,“in communication with” and the like, may be used to indicate electrical or data communication, which may be accomplished by physical contact, induction, electromagnetic radiation, radio signaling, infrared signaling or optical signaling, for example. One having ordinary skill in the art will appreciate that multiple components may inter-operate and modifications and variations are possible of achieving the electrical and data communication.

In some embodiments described herein, the term“coupled,”“connected,” and the like, may be used herein to indicate a connection, although not necessarily directly, and may include wired and/or wireless connections.

Note that although terminology from one particular wireless system, such as, for example, 3GPP LTE and/or New Radio (NR), may be used in this disclosure, this should not be seen as limiting the scope of the disclosure to only the aforementioned system. Other wireless systems, including without limitation Wide Band Code Division Multiple Access (W CDMA), Worldwide Interoperability for Microwave Access (WiMax), Ultra Mobile Broadband (UMB) and Global System for Mobile Communications (GSM), may also benefit from exploiting the ideas covered within this disclosure.

Note further, that functions described herein as being performed by a wireless device or a network node may be distributed over a plurality of wireless devices and/or network nodes. In other words, it is contemplated that the functions of the network node and wireless device described herein are not limited to performance by a single physical device and, in fact, can be distributed among several physical devices.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

In general, methods and systems of an SLA prediction model for predicting the SLA for microservice-based applications is disclosed. These applications are composed of loosely coupled services and their interaction may negatively impact the overall performance of the service. The inventive methods and systems are based on a Reinforcement Learning (RL) based Genetic Algorithm (GA) in order to handle a large number of microservices. These methods and systems are composed of both an offline and online portion working in conjunction. The offline portion tests the offline SLA conformance results for the selected feature sets and these results serve as training data to create an SLA prediction model. The resulting prediction model is then fed into one or more online SLA predictors. The online SLA conformance results are fed back to the offline portion which may cause the feature sets to mutate, update the SLA conformance testing and improve the offline SLA prediction.

The methods and systems of an SLA prediction model as disclosed herein provide various advantages over the prior art. One such advantage includes the GA and reinforcement learning being combined for a long-lived learning mechanism for Feature Compositions (FCs) validation. The reinforcement learning modifies the GA mutation process from a randomly-triggered operation to a feedback-triggered operation which avoids the cost of testing a large number of randomly composed features.

Additionally, the GA is slow to achieve optimal FCs and therefore, to improve the GA performance the methods and systems disclosed herein executes microservice- based offline FC testers in parallel and uses the execution history as input to accelerate the GA evolution. Further, the offline and online feature composition testing are combined. The offline portion accelerates the FC selections while the online portion provides feedback on how the FCs actually perform.

Further, the feature configuration, resource usage and service level indicators are added as genes to the Feature Composition (FC) chromosomes to increase the accuracy of the prediction. Additionally, the GA is selected since it can handle large scale problems and is more scalable than other approaches used to do SLA prediction. Still further, relevant feature composition policies are applied to reduce the dimension of the FC set to be tested and predicted. The FC set would otherwise include all possible combinations of the existing features.

Referring now to the drawing figures, in which like elements are referred to by like reference numerals, there is shown in FIG. 1 a schematic illustration of an embodiment of a system for the prediction of feature composition SLA prediction and is composed of two parts that run in parallel, namely, an offline 101 portion and an online portion 103. The system utilizes a reinforcement learning (RL) based genetic algorithm (GA) for validating new and existing service compositions.

In some embodiments, the offline portion 101 is responsible for creating and testing an offline subset list of valid feature compositions (FCs) 152 and for the training of an offline SLA predictor trainer (SPT) 116 based on the offline subset list of valid feature compositions 152 post-testing, whereby the SPT 116 creates an offline SLA prediction model 160. The online portion 103 is responsible for evaluating an online SLA prediction result 164 for a real-time requested FC 162 based on the offline SLA prediction model 160 and for providing the online SLA conformance result 156 to the offline portion 101 for inclusion with further creating, testing and training. The offline portion 101 includes an offline feature composition generator (FCG) 102 that is configured to create an offline list of valid feature compositions (FCs) 105. The offline list of valid FC 105 is stored in the Feature Composition (FC) Repository 114.. The FC Repository 114 is a storage that maintains relative information for FC, e.g. FC population, which includes the associated configurations; SLI, e.g. number of requests per second, latency, throughput, etc.; and expected resources, e.g. compute, storage, as well as the resulting SLA status, e.g. comply or violate. Each FC is based on a feature set, which includes the added feature configuration 150 which is composed of a feature, its configuration as well as the required resources, as well as any relevant composition policies 110 and system SLA requirements 112.

The inclusion of the composition policies 112 serve to reduce the list of valid FCs 152 and are defined by the operator. This allows the list of valid FCs 152 to be shortened without affecting the performance of the system 100. For example, for a set of features fl, f2, ... fh, if f2 and f4 are mutually exclusive, there is no need to test and predict SLA conformance for FCs that include both f2 and f4.

In some embodiments, the FCG initially creates the list, or population, of FCs by applying a genetic algorithm (GA) and which is then restricted by the composition policies 112. When applying the GA, the FCG may perform the GA operations of selection, crossover and mutation. The resulting list of valid FCs 152 is defined as a set of features, configurations, expected resources and service level indicators, e.g. number of requests per second, latency, throughput, etc.

The FCG inputs an offline subset list of valid FCs 152 to a composition test manager (CTM) 104 which in turn initiates the testing of the offline list of valid FCs 152 by one or more composition testers (CT) 106, 108. This list is not necessarily the entire list, or population, of valid FCs, but new or modified FCs created due to the GA operations of crossover or mutation. For example, if a new feature is added or FCs mutated due to online results, those new FCs would be tested but the remaining unaffected FCs would not.

The CT test each of the FCs to determine the offline SLA conformance results 158, sometimes referred to as the offline SLA status. These results 158 may be either compliant or non-compliant/violation for each FC. Following the testing, the CTs 106, 108 will send the offline SLA conformance result for each FC to the CTM 104, which will in turn be sent to the FCG 102. The inclusion of two CTs 106, 108 is illustrative and not meant to be limiting. Those skilled in the art will recognize that multiple CTs may be utilized and are within the scope of the present invention.

After receiving the offline SLA conformance results 158 from the CTM 104, the FCG 102 is configured to provide the offline SLA conformance results 158 to an offline SLA predictor trainer (SPT) 116 to be used as training data. The SPT 116 is trained and creates an SLA prediction model 160. A machine leaning algorithm, such as, but not limited to, Recurrent Neural Network (RNN), can be used to train the SPT. In some embodiments, the testing resources are released once the testing is complete and the required training resources decrease as the GA model matures with a better SLA prediction accuracy for the FCs.

Following training, the SPT 116 is configured to provide the SLA prediction model 160 to one or more online SLA predictors 118 located within the online portion 103 to determine an online SLA conformance result 156 for a real-time requested FC SLA prediction 162.

In some embodiments, the online portion 103 includes one or more SLA predictors 118 that are configured to evaluate whether a real-time online FC request 162 would be comply or not to the SLA based on the offline SLA prediction model 160 and the current available resources 167. If the SLA predictor 118 determines the online FC request 162 would result in an SLA violation, it will reject 122 the FC request 162. If the SLA predictor 118 determines the FC request 162 would result in SLA compliance 164, the FC request will be allowed.

Upon allowance, a feature composition execution 124 will execute the FC request. A system monitor 120 provides online data 166, e.g. the current, or actual, resource usage for the allowed FC request.

Upon feature composition execution completion 124, an online system SLA analytics 126 will report an online SLA conformance result 156, that includes an SLA fulfillment status for the real-time requested FC, together with the online data 166, e.g. actual SLI (slF) and actual resources (r’) used by the FC request, to the offline FCG 102. The FCG 102 uses the online SLA conformance results 156 and online SLA data 166 in the updating of the offline list of valid FCs 152.

In some embodiments, the FCG 102 may perform the GA operation of mutation on the FCs based on both the offline test results and online execution results. For example, for an SLA violation sample, the system SLA analytics 126 notifies the FCG 102 with the actual resources (r’), actual SLI (slF) and SLA result. The FCG 102 will determine if mutation is required and if so, create a new chromosome with the actual values. The FCs are modified/mutated accordingly, and these new FCs will trigger additional offline testing by the CTs 106, 108.

In some embodiments, the FCG 102 performs the GA operation of mutation once the difference between the required resources and service level indicators with the actual resources and actual service level indicators meets established thresholds. The thresholds are set for the actual resources and SLI for a particular FC in order to avoid continual mutation of FCs by FCG 102. For example, the number of requests per second may be set to 100 requests per second plus/minus 3 requests. In such a situation, if the actual value is less than 97 or greater than 103, then the mutation operation takes place. However, if the actual value is between 97 and 103, the mutation operation will not take place.

In some embodiments, the online portion 103 includes an FC manager that will not enable FCs unless they have been tested offline.

The Feature Composition Generator 102 utilizes a Genetic Algorithm (GA).

In general, a GA is a process inspired by the process of natural selection that belongs to the larger class of evolutionary algorithms (EA). Genetic algorithms are commonly used to generate high-quality solutions to optimization and search problems by relying on bio-inspired operators such as mutation, crossover and selection. In a GA, a population of candidate solutions (called individuals, creatures, or phenotypes) to an optimization problem is evolved toward better solutions. Each candidate solution has a set of properties (its chromosomes or genotype) which can be mutated and altered.

The evolution usually starts from a population of individuals, and is an iterative process, with the population in each iteration called a generation. In each generation, the fitness of every individual in the population is evaluated; the fitness is usually the value of the objective function in the optimization problem being solved. The more fit individuals are stochastically selected from the current population, and each individual's genome is modified, through repetitive application of the mutation, crossover, inversion and selection operators, to form a new generation. The new generation of candidate solutions is then used in the next iteration of the algorithm. Commonly, the algorithm terminates when either a maximum number of generations has been produced, or a satisfactory fitness level has been reached for the population.

In some embodiments, the FCG 102 utilizes the Genetic Algorithm (GA) where genes are defined as the features with associated configurations and required resources as well as the service level indicators (SLI); and the chromosomes are the resulting FCs. These are of variable length but are bound by the number of features/microservices per FC. The population is a set of chromosomes, that is, the FC repository. This repository may grow as new features are added, and new FCs defined (i.e. add new features to the existing FCs). The population remains relatively constant since it is restricted by the composition policies.

In these embodiments, the FCG 102 performs the FC selection, crossover and mutation operations. The crossover serves to generate new offspring in the FC population. The crossover operation is not randomly chosen but is restricted by the composition policies defined by an operator. The mutations occur when the FCs are transformed with the actual resources and actual service level indicators for actual feature with actual configuration. These are existing FCs updated with offline CTs and real-time/online results. The fitness function is the resulting accuracy of the offline SLA prediction. The stopping criteria is when the accuracy for the offline FC SLA prediction is deemed acceptable (i.e. SLA conformance).

As illustrated in FIG. 2, an exemplarily example of the creation of an initial list of Feature Compositions (FCs) is disclosed. In this exemplarily example, the FCs are formed based on a combination of microservice/features, resources and SLI, in view of relative composition policies. For example, FCi may be based on microservice/features A2, B7 and C3, along with resource requirements R4 and SLI Si (FCi = A₂, By, C₃, R4, SI).

FIG. 3 provides an example of a one-point crossover operation of a pair of parent FCs. The rightmost part/genes are swapped between the FC parents FCi and FC2 to produce the new FCi and FC2 offspring. For example, the parent FCi originally includes A2, B7, C3, R4 and Si as genes and the parent FC2 originally includes Ai, B2, Cs, Ri and S7 as genes. Following the one-point crossover operation, offspring FCi includes A2, B7, Cs, Ri and S7 as genes and the offspring FC2 includes Ai, B2, C3, R4 and Si as genes. The use of a one-point crossover operation is illustrative and not meant to be limiting as there are several types of crossover operators, e.g. one-point, multi-point, uniform, etc. Those skilled in the art will recognize the other types of crossover operation maybe used and are within the scope of the present invention.

Further, as the FCs must be valid, e.g. must satisfy specific composition policies, the resulting offspring FCs are evaluated in light of relevant composition policies. For example, as illustrated in FIG. 4, a composition policy restricts having feature A2 with B2. For a two-point crossover operation of a pair of parent FCs, the middle part/genes are swapped, between the FC parents FCi and FC2 to produce the new FCi and FC2 offspring. For example, the parent FCi originally includes A2, B7, C3, R4 and Si as genes and the parent FC2 originally includes Ai, B2, Cs, Ri and S7 as genes. Following the two-point crossover operation, offspring FCi includes A2, B2, Cs, R4 and Si as genes and the offspring FC2 includes Ai, B7, C3, Ri and S7 as genes. However, when offspring FCi and FC2 are compared to the composition policy, FCi is determined to be invalid since A2 and B2 cannot be combined.

In some embodiments, the mutation of FCs is performed. In these embodiments, simple replacement of one or more genes of a FC with genes obtained offline and/or online, such as the replacement of the estimated resources and SLI with the actual resources and SLI values obtained from the offline CTs and online SLA Analytics. FIG. 5 provides an example of a mutation operation of a FC. In this example, the original FC includes A2, B7, C3, R4 and Si as genes. Following the mutation operation where the resources gene and the SLI values gene are replaced, the resulting FC includes A2, B7, C3, R3 and Ss as genes. The use of a simple replacement mutation operation is illustrative and not meant to be limiting as there are several types of crossover operators, e.g. bit flip mutation, random resetting, swap mutation, scramble mutation, inversion mutation, etc. Those skilled in the art will recognize the other types of mutation operation maybe used and are within the scope of the present invention. FIG. 6 illustrates an exemplary offline FC SLA prediction procedure. The FC SLA prediction procedure may be utilized when there is an addition of a new feature/microservice to an existing service. The operations shown in the exemplary procedure of FIG. 6 are labelled numerically, but this numbering is used only to facilitate clarity in the following description. Moreover, the numerical order is merely exemplary, and the order of the various operations can be rearranged in certain embodiments. Any dashed lines indicate optional operations whose performance may depend on one or more conditions.

In operation 1, when a new feature is added for a service, an add feature request is sent to the FCG. The request includes the added feature along with any associated configuration and required resources. The FCG selects FCs based of the FC composition policies and adds the new feature (i.e. gene) with required resources to the selected FCs. This may also initiate crossover procedure of selected FCs. The new FCs are stored in the FC repository.

In operations 2 and 3, the FCG sends a test FC request the Composition Test Manager (CTM) for the new offspring FCs. The CTM defines the FC tests to be executed based on the service level indicators (SLI) and feature resources. Operation 2 refers to the first test FC request sent relating to the first new offspring FC.

Operation 3 refers to the N^th test FC request relating to the N^th new offspring FC, in the event there are two or more new offspring FCs.

In operations 4 and 5, the test FC requests initiate the testing of the new offspring FCs by offline Composition Testers (CTs). The CTs test the offline SLA conformance, e.g. throughput, of these FCs. The offline SLA conformance results may be either an SLA conformance, i.e. SLA compliant, or an SLA violation, i.e.

SLA non-compliant, based on the defined SLIs. For example, the SLIs may specify performance metrics such as throughput or latency.

Operation 4 refers to the request to initiate the testing of the first new offspring FC by a first CTi. Operation 5 refers to the request to initiate the testing of the N^th new offspring FC by a Nth CTN, in the event there are two or more new offspring FCs.

In Operation 6 and 7, the offline SLA conformance results for each of the new offline FCs, e.g. FCi ... FCN, are sent to the CTM from the CTs, e.g. CTi ... CTN. In Operation 8 and 9, the offline SLA conformance results for each of the new offline FCs, e.g. FCi ... FCN, are sent to the FCG from the CTM. The FCG updates the FCs in the repository with offline SLA conformance results. These results are feed to the SPT which in turn, updates the online SLA predictor.

In operation 10, the resulting offline SLA conformance result is provided to the SLA Prediction Trainer (SPT) to serve as training data for the SPT. The SPT leams the FCs. A machine learning algorithm is used to train the SPT.

In operation 11, once the SPT leams the FC SLA conformance results, the SPT updates the online SLA Predictor.

Following the update of the online SLA predictor, a user’s request for execution of an application/service will be translated into a prediction request sent to the SLA predictor for a Feature Composition of microservices (e.g. FCi). Based on the training data (e.g. FCi SLA Results and FC4_SLA_Results), the current actual resources and SLI, the SLA Predictor predicts the probability of SLA violation. If no violation is predicted, the service may execute.

The Service executes, and the Systems SLA Analytics gathers the actual resources, SLI and SLA conformance results. It sends this information back to the FCG which may in turn, decide to mutate the FC (e.g. FCi). This initiates new CTs that generate new training data for the SLA Prediction Trainer.

Figure 7 illustrates an exemplary FC mutation procedure. The FC mutation procedure may be utilized when online SLA conformance results differ from offline results. The operations shown in the exemplary procedure of FIG. 7 are labelled numerically, but this numbering is used only to facilitate clarity in the following description. Moreover, the numerical order is merely exemplary, and the order of the various operations can be rearranged in certain embodiments. Any dashed lines indicate optional operations whose performance may depend on one or more conditions.

In operation 1, the online system SLA analytics sends the online SLA conformance results for a real-time requested FC to the FCG. The online SLA conformance results may include the actual configuration, actual resources and actual SLI used by the real-time requested FC. This may initiate a mutation of the FC by the FCG. In some embodiments, operation 1 includes threshold values that are used to determine whether a mutation is needed or not required in order to avoid continuous updating/mutating offline FCs due to slight variations in the actual parameters and SLA conformance. In such embodiments, a mutation is not performed when the variation, or difference, between the offline SLA conformance results and the online SLA conformance results are minimal.

In operation 2, FCG sends a test FC request to the Composition Test Manager (CTM) for the new mutated FC. The CTM will then define the FC tests to be executed based on the service level indicators (SLI) and feature resources for the mutated FC.

The procedure continues with operation 4 set out in FIG. 6 above.

As will be appreciated by one of skill in the art, the concepts described herein may be embodied as a method, data processing system, and/or computer program product. Accordingly, the concepts described herein may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a “circuit” or“module.” Furthermore, the disclosure may take the form of a computer program product on a tangible computer usable storage medium having computer program code embodied in the medium that can be executed by a computer. Any suitable tangible computer readable medium may be utilized including hard disks, CD-ROMs, electronic storage devices, optical storage devices, or magnetic storage devices.

Some embodiments are described herein with reference to flowchart illustrations and/or block diagrams of methods, systems and computer program products. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable memory or storage medium that can direct a computer or other

programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

It is to be understood that the functions/acts noted in the blocks may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality /acts involved. Although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that

communication may occur in the opposite direction to the depicted arrows.

Computer program code for carrying out operations of the concepts described herein may be written in an object-oriented programming language such as Java® or C++. However, the computer program code for carrying out operations of the disclosure may also be written in conventional procedural programming languages, such as the "C" programming language. Machine learning models can be implemented using for example, R or Python libraries. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer. In the latter scenario, the remote computer may be connected to the user's computer through a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate every combination and subcombination of these embodiments. Accordingly, all embodiments can be combined in any way and/or combination, and the present specification, including the drawings, shall be construed to constitute a complete written description of all combinations and subcombinations of the embodiments described herein, and of the manner and process of making and using them, and shall support claims to any such combination or sub-combination.

The following abbreviations are explained:

Abbreviation Explanation

ANA System SLA Analytics

CD/CI Continuous Delivery /Continuous Integration

CT Composition Testers

CTM Composition Test Manager

FC Feature Composition

FCG Feature Composition Generator

GA Genetic Algorithm

OSP Online SLA Predictor

SPT SLA Prediction Trainer

It will be appreciated by persons skilled in the art that the embodiments described herein are not limited to what has been particularly shown and described herein above. In addition, unless mention was made above to the contrary, it should be noted that all of the accompanying drawings are not to scale. A variety of modifications and variations are possible in light of the above teachings without departing from the scope of the following claims.

Claims

CLAIMS What is claimed is:

1. A method to validate Feature Composition (FC), the method comprising the steps of:

creating an offline list of valid Feature Compositions (FCs);

testing offline the list of valid FCs to determine offline SLA conformance results for each of the FCs of the list of valid FCs;

providing the offline SLA conformance results to an offline SLA predictor trainer (SPT) to be used as training data;

training the SPT using the training data to create an SLA prediction model; providing the SLA prediction model to one or more online SLA predictors to determine an online SLA prediction result for a real-time requested FC

testing and evaluating the online SLA prediction result to determine an online SLA conformance result; and

providing the online SLA conformance result to the offline FCG, wherein the FCG uses the online SLA conformance result in the updating of the offline list of Feature Compositions (FCs).

2. The method of claim 1, wherein the offline SLA conformance results are defined as an offline list of Service Level Agreement (“SLA”) compliant FCs and an offline list of Service Level Agreement (“SLA”) non-compliant FCs.

3. The method of claim 1, wherein the creating the offline list of valid FC comprises inputting a list of FCs into a Feature Composition Generator (FCG), the FCG utilizing a Genetic Algorithm (GA) to create a list of FCs, the FCG applying composition policies to the list of FCs to create a list of valid FCs.

4. The method of claim 3, wherein the FCG performs the GA operations of selection, crossover and/or mutation on the list of FCs

5. The method of any claims 1- 4, wherein each of the FCs in the offline list of valid FCs is defined as including a feature set, associated configurations, required resources and service level indicators.

6. The method of any claims 1-5, wherein the testing offline of the list of valid FC comprises the following steps:

inputting the list of valid FCs into a Composition Test Manager (CTM); providing, by the CTM, the list of valid FCs to one or more offline

Composition Testers (CT); and

testing each of the FCs of the list of valid FCs by the one or more CTs to determine offline Service Level Agreement (“SLA”) conformance results for each of the FCs.

7. The method of claim 6, further comprising:

providing, by the CTM, the offline SLA conformance results for each of the FCs to the FCG; and

updating, by the FCG, the list of valid FCs based on the offline SLA conformance results.

8. The method of any claims 1-7, further comprising:

training the offline SPT using a machine learning algorithm using the training data to create a prediction model.

9. The method of claim 8, wherein the machine learning algorithm is a Recurrent Neural Network (RNN) algorithm.

10. The method of any claims 1-9, further comprising the step of:

allowing or prohibiting the real-time requested FC based on the online SLA prediction result and the current available resources.

11. The method of any claims 1-10, wherein the testing of the online SLA prediction result to performed by a Feature Composition Execution, and wherein the evaluation of the online SLA prediction result is performed by a System SLA

Analytics, to determine an online SLA conformance result online SLA prediction result.

12. The method of any claims 1-11, wherein the online SLA conformance result includes an SLA fulfillment status for the real-time requested FC, and the actual service level indicators (SLI) and actual resources used by the real-time requested FC.

13. The method of Claim 12, wherein the FCG performs the GA operation of mutation on the list of FCs to include actual service level indicators and actual resources during the updating of the list of FCs.

14. The method of Claim 13, wherein the FCG performs the GA operation of mutation once the difference between the required resources and service level indicators with the actual resources and actual service level indicators meets an established threshold.

15. A system to validate feature composition (FC), the system comprising:

an offline feature composition generator (FCG) (102) configured to create an offline list of valid feature compositions (FCs) (105);

a composition test manager (104) configured to test offline a subset list of valid FCs (152) from the offline list of valid FCs (105) to determine offline SLA conformance results (158) for each of the FCs of the subset list of valid FCs;

the FCG (102) configured to provide the offline SLA conformance results (158) to an offline SLA predictor trainer (SPT) (116) to be used as training data; the SPT (116) configured, using the training data, to create an SLA prediction model (160);

the SPT (116) configured to provide the SLA prediction model (160) to one or more online SLA predictors (118) to determine an online SLA prediction result (164) for a real-time requested FC SLA prediction (162); and a system SLA analytics (126) configured to provide the online SLA conformance result (156) to the FCG (102), wherein the FCG (102) uses the online SLA conformance result (156) in the updating of the offline list of valid FCs (105).

16. A method to validate Feature Composition (FC), the method comprising the steps of:

creating an offline list of valid Feature Compositions (FCs);

providing the offline list of valid Feature Compositions (FCs) for testing to determine offline SLA conformance results for each of the FCs of the list of valid FCs;

receiving the offline SLA conformance results; and

providing the offline SLA conformance results to an offline SLA predictor trainer (SPT) to be used as training data.

17. The method of claim 16, the method further comprising the steps of:

receiving online SLA conformance results; and

updating the offline list of valid Feature Compositions (FCs) based on the online SLA conformance results.

18. The method of any of the claims 16-17, wherein the offline SLA conformance results are defined as an offline list of Service Level Agreement (“SLA”) compliant FCs and an offline list of Service Level Agreement (“SLA”) non-compliant FCs.

19. The method of any of the claims 16-18, wherein the creating the offline list of valid FC comprises inputting a list of FCs into a Feature Composition Generator (FCG), the FCG utilizing a Genetic Algorithm (GA) to create a list of FCs, the FCG applying composition policies to the list of FCs to create a list of valid FCs.

20. The method of claim 19, wherein the FCG performs the GA operations of selection, crossover and/or mutation on the list of FCs.

21. The method of any claims 16-19, wherein each of the FCs in the offline list of valid FCs is defined as including a feature set, associated configurations, required resources and service level indicators.

22. The method of any of the claims 16-21, further comprising:

updating the offline list of valid FCs based on the offline SLA conformance results.

23. The method of Claim 22, wherein the FCG performs the GA operation of mutation on the offline list of FCs to include actual service level indicators and actual resources during the updating of the list of FCs.

24. The method of Claim 23, wherein the FCG performs the GA operation of mutation once the difference between the required resources and service level indicators with the actual resources and actual service level indicators meets an established threshold.

25. A system to validate feature composition (FC), the system comprising:

an offline feature composition generator (FCG) (102) configured to perform any operations of any of the claims 16-25.

SPT focused

26. A method to train an offline SLA predictor trainer (SPT), the method comprising the steps of:

receiving offline SLA conformance results to be used as training data;

training the SPT using a machine learning algorithm using the training data to create a prediction model.

27. The method of claim 26, wherein the machine learning algorithm is a Recurrent Neural Network (RNN) algorithm.

28. A system to validate feature composition (FC), the system comprising:

an offline SLA predictor trainer (SPT) (116) configured to perform any operations of any of the claims 26-27.

29. A method to validate Feature Composition (FC), the method comprising the steps of:

receiving a SLA prediction model to one or more online SLA predictors to determine an online SLA prediction result for a real-time requested FC; and

testing and evaluating the online SLA prediction result to determine an online SLA conformance result.

30. The method of claim 29, further comprising the step of:

allowing or prohibiting the real-time requested FC based on the online SLA prediction result and any current available resources.

31. The method of any claims 29-30, wherein the testing of the online SLA prediction result to performed by a Feature Composition Execution, and wherein the evaluation of the online SLA prediction result is performed by a System SLA

32. The method of any claims 29-31, wherein the online SLA conformance result includes an SLA fulfillment status for the real-time requested FC, and the actual service level indicators (SLI) and actual resources used by the real-time requested FC.

33. A system to validate feature composition (FC), the system comprising:

one or more online SLA predictors (118) configured to perform any operations of any of the claims 29-32.