WO2014048851A2

WO2014048851A2 - A method and a system to surveil applications of online services

Info

Publication number: WO2014048851A2
Application number: PCT/EP2013/069601
Authority: WO
Inventors: Fermín GALÁN; Ignacio BLASCO; Daniel MORÁN; José Juan SÁNCHEZ; Alvaro MARTÍNEZ REOL
Original assignee: Telefonica SA
Current assignee: Telefonica SA
Priority date: 2012-09-25
Filing date: 2013-09-20
Publication date: 2014-04-03
Anticipated expiration: 2015-03-25
Also published as: WO2014048851A3

Description

A method and a system to surveil applications of online services

Field of the art

The present invention generally relates, in a first aspect, to a method to surveil applications of online services, and more particularly to a method for monitoring a web application in order to check and control its operation while providing online services.

A second aspect of the invention relates to a system adapted for implementing the method of the first aspect.

Prior State of the Art

Most of the online services existing today are based on applications (e.g. online web applications). Users typically access to such web applications using conventional browsers from their computing devices (workstations, laptops, smartphones, etc.) to get a given service from them, e.g. book in a hotel booking system, by an item in an online shop, buy a flight ticket in a travel agency portal, etc. These web applications can serve to internal users in a given organization (e.g. the intranet portal of a given company) or can be publicly accessible through the Internet. Some applications are used by a huge number of concurrent users (even millions); in which case the application has to be designed to support such load.

Except for very simple applications, they allow some degree of arbitrary interaction, e.g. the application has several options ("search for a flight", "purchase a ticket", "show previous purchases", etc.) each one needing data (in web forms, etc.) that would be different each time (the search option allow to specify the destination in a combo box, so an user could introduce a different city each time he/she uses the application). By navigation sequence its understood a particular interaction on the application, from the moment the user accesses to the application until he/she finished, including the sequence of all the transactions done and the data used in each one.

Applications need computational, storage and network resources to work, so they are deployed and run in Information Technology (IT) infrastructure (either "conventional" datacenters or, more recently, in cloud platforms) where these resources lay. The amount of resources needed to perform properly depends on the particular application and the load it supports. In other words, the more users are using the application (and more resources-intensive are the navigation sequences they are performing), the more resources the application would need. An application with under-sized resource would perform poorly or not perform at all. This is not a desired situation for the application owner, as it could have severe business impact, e.g. employees cannot do their work (case of an internal application) or users will stop using the application and look for the same service in a competitor (case of public applications).

Thus, the performance of any application with an associated (fixed) amount of resources would be dependent of the load. The problem is that load cannot be predicted, it is not possible to know a priori how much users would be interacting with the application in a given moment and which particular navigation sequences would they be performing. So, it is needed to continuously surveil the performance of the running application, so its owner can anticipate performance degradation and take the appropriated corrective actions in that case (e.g. allocate more resources to the application to cope with the growing load).

Related technologies in the field are monitoring systems (such as Nagios, Ganglia, Zenoss Unified Monitoring or Zabbix), load injectors (such as JMeter) and web automation tools (such as Selenium).

Problems with existing solutions:

Existing monitoring solutions (Nagios, Ganglia, Zenoss or Zabbix) are mainly focused not in monitoring the application itself, but in monitoring the systems where the application is deployed. That is, these tools are oriented to monitoring Central Processing Unit (CPU) load, Random Access Memory (RAM) allocation, disk Input/Output (I/O), node network availability (through ping) and other low level parameters. However, application performance has to be measured in terms of transaction time (i.e. how much time the application is able to serve a given transaction, the lower the better) or transactions per second. In other words, from the point view of the user it is not relevant the CPU load (or any other low level metric). In fact, user has no visibility of any of such metric. What is important for the user is to be able to perform his/her transactions or a sequence of transactions (i.e. a navigation sequence) quickly and efficiently. And, at the end, the performance of the application cannot be derived from the low level performance metrics. A given application CPU and Random Access Memory (RAM) could be ok but the application could not perform correctly for other reasons, and vice versa.

Load injection tools (such as JMeter) can be used to study the performance of an application under controlled circumstances, but are more oriented to stressing the application with synthetic load than to monitoring it in real usage conditions. In fact, using a load injector tool in a production application is a bad idea, as it would heavily interfere in the normal behavior of the application and could impact performance.

Some other tools have some form of application monitoring, in the sense they can check the availability of a given Transmission Control Protocol (TCP) port or even query a Uniform Resource Locator (URL) with a HyperText Transfer Protocol (HTTP) request. The most advanced in this sense is probably Selenium, a web automation framework. However, it only covers how to define the navigation sequence and reproduce it, and lacks important features like:

• The specification of a quality limit for the application

• The calculation of a reference reproduction time without using external tools.

• Actions derived of the result of reproducing the navigation sequence or combinations of navigation sequences Summary of the Invention

An object of the present invention is to eliminate the drawbacks mentioned above by providing an application surveillance system that allows application owners to check the performance of their applications based on navigation sequences that actually represent the behavior of the users on the application instead of using CPU load, RAM consumption or any other low level metric. In this sense, the invention includes a method and a system to define navigation sequences, application quality limits, navigation combination and the triggering of actions based on surveillance.

The invention includes advanced features such autocalibration loop for navigation sequences, navigation sequence anomalies detection, quality limits recommendations, navigation sequences combination and flexible action triggering with success check.

The present invention relates, in a first aspect, to a method to surveil applications of online services, comprising as commonly used in the field monitoring at least one web application in order to check and control operation of said at least one web application, said at least one web application deployed and running in an internal or external controlled Information Technology (IT) infrastructure and adapted to be accessed for at least one user device with computing capacity.

On contrary to the known proposals, the method of the first aspect comprises acquiring, by means of a surveillance application including a frontend unit, a database and a recording unit, a plurality of navigation sequences representing the behavior of said at least one user with said at least one web application while providing an online service, said surveillance application interacting with at least one user device with computing capacity.

The surveillance application comprises recording by said recording unit each one of said plurality of navigation sequences and storing in said database said recorded navigation sequences. Communication is provided by means of a plurality of interfaces about interactions of said surveillance application with a programmable probe system, a scaling and healing system and a notification system external to said surveillance application.

A calibration of said stored and recorded navigation sequences is also performed by choosing any sequence of said recorded navigation sequences.

Moreover, a quality limit is also defined for each one of said plurality of navigation sequences and a triggering condition is defined based on a combination of said plurality of sequences.

Finally, the method also comprises checking the success or failure of said triggering.

Other embodiments of the method of the first aspect of the invention are described according to appended claims and in a subsequent section related to the detailed description of several embodiments.

A second aspect of the present invention is related to a system to surveil applications of online services, said system surveiling online services comprising:

at least one user device with computing capacity to access to at least one web application;

said at least one web application deployed and arranged to run in an internal or external controlled Information Technology (IT) infrastructure and adapted to be accessed for said at least one user device with computing capacity, and

a plurality of computational, storage and network resources provided by said internal or external controlled IT infrastructure.

On contrary to the known proposal the system of the second aspect comprises a surveillance application adapted to control said at least one web application in order to acquire a plurality of navigation sequences representing the behavior of said at least one user with said web application while providing an online service.

In an embodiment the surveillance application comprises:

- a frontend unit arranged to provide a user interface to said at least one user device with computing capacity using said system;

- a database to store a plurality of navigation sequences with associated information;

- a recorder unit arranged to record said stored plurality of navigation sequences;

- a plurality of interfaces arranged to establish communication with a probe system, a scaling and healing system a said notification system. The system of the second aspect is adapted to implement the method of the first aspect.

Brief Description of the Drawings

The previous and other advantages and features will be more fully understood from the following detailed description of embodiments, with reference to the attached, which must be considered in an illustrative and non-limiting manner, in which:

Figure 1 is an Applications Surveillance System controlling a set of running applications deployed and running on an external and controlled IT infrastructure, according to an embodiment.

Figure 2 is the Applications Surveillance System controlling a set of running applications deployed and running on an internal and controlled IT infrastructure, according to an embodiment. Detailed Description of Several Embodiments

Figure 1 and Figure 2 show embodiments of the Applications Surveillance System in which the invention is based. More specifically, the system is controlling a set of running applications deployed and running on an internal and controlled IT infrastructure (Figure 2) (e.g. a cloud computing platform) or in an external and controlled IT infrastructure (Figure 1 ). The particular applications are not relevant as far as each given application exposes a navigable interface through a network (such as the Internet or an intranet). It means by navigable interface a set of interlinked web pages and the controls to navigate and operate them (links, forms, buttons, etc.). The Applications Surveillance System is able to interact with a programmable Probe System, a Scaling System and a Notification System. Users of the Applications Surveillance System are supposed to be managers of the applications (each user is the manager of one or more applications).

The Application Surveillance System is composed of the following units or modules:

• Frontend. This unit provides the user interfaces for the user of the system, implementing the methods described below. It deals with user authentication, authorization (e.g. each user only can apply the methods to the applications he/she owns) and method selection.

• Database. It stores navigation sequences with associated information

• Recorder. Used to record navigation sequences. • Interfaces. Several modules in charge of communication with the Probe

System, the Scaling and Healing system and the Notification System.

The Application Surveillance System implements the following actions or methods:

A Record navigation sequence:

1 . The Frontend offers a list of the applications managed by that user. The user selects one of them or specifies the entry point (home page) of an external application.

2. The user is presented the home page of the selected application, accessed by the Application Surveillance System through the Recorder.

3. The user signals the start of the recording process (e.g. press a "start button" in some part of the Frontend) and navigates the application as he/she desires, resembling a real user of the application. The Recorder performs and records each one of the navigation steps in a navigation sequence (i.e. and ordered list of steps executed on the application during the navigation session). Once he/she finished, he/she signals the stop of the recording (e.g. press a "stop button" in some part of the Frontend).

^■ Let T_G be the time from the starting signal and the stop signal.

^■ Let T_A be the effective navigation time, recorded by the Recorder system subtracting to T_G the "thinking times" (T_T) during which the user is not interacting with the application. Thus, T_G = T_A + T_T.

4. The navigation sequence is stored in the database associating <T_G, T_A>. The identifier of the application in which context the navigation sequence has been recorded is also stored.

B Calibrate navigation sequence:

1. The Frontend checks database and offers a list of the navigation sequences recorded by that user (according to procedure A). The user chooses a sequence, which associated <T_G, T_A,>.

2. The sequence is passed to the Probe System for reproduction. The Probe System reproduces the sequence several times and takes the mean value of the reproduction time (T_R) using one of the following procedures:

a) The sequence is reproduced N times, being N a fixed value. b) The sequence is reproduce as many times (Λ/) as possible within a given maximum time T_max. If N is lesser than a minimum acceptable number (N_m(1) then the user is reported, so it can cancel the recording or continue until reach N_m/(1 samples.

c) The sequence is reproduced as many times needed so the mean value converges within a given internal ε for N samples, being N a fixed value, which a minimum of m (being m fixed value). A maximum of M (being M a fixed value) is defined, so if the sequence does not converge after M samples, a problem is reported to the user ("sequence is unstable").

d) A variation of c) in which M is not a fixed value, but a maximum time T_max. If the sequence does not converge after reproducing it during T_max, a problem is reported to the user ("sequence is unstable").

3. If T_R > ccT_G (a is a correction factor, 0<a≤1, a = 1 by default) then a problem is reported to the user (given that should be T_R < T_G always, due to the thinking times).

4. The T_R time is stored in the database associated to the sequence (so the information associated to the sequence should be at the end <T_G, T_A, T_R>).

Start application surveillance:

1. The Frontend offers a list of the applications for which navigation sequences have been recorded. The user selects one of them.

2. The user is presented a list of the navigation sequences, it is assumed that the user has recorded at least one navigation sequence, recorded and calibrated by that user (according to procedures A and B) for the chosen application in the previous step. The user chooses a subset of sequences (subset could be just one sequence), each one with its associated information (<T_G, T_A, T_R>).

3. For each one of the sequences (being /^' each particular sequence), the user defines a quality limit using one of the following options:

^■ An absolute value for the target user perceived navigation time

^■ A percentage (p) of T_Gj for the target user perceived navigation time, so T'_Gj = p^■ T_G >. ^■ A degradation factor (t/,)

^■ A recommendation done by the Application Surveillance System for either u, or T'_Gj. This recommendation could be based on the mean value among time of the measured time for the sequence by the Probe System with a security margin.

Whatever the case, an effective time threshold (L/,) is defined using one of the following formula:

U = T_R(l + u), u = ^—^L - l

^LA

The user defines the triggering condition based on the combination of sequences within the sequences subset

^■ Trigger action when all sequences overpass their threshold

^■ Trigger action when at least one sequence overpass its threshold

^■ Trigger action when a given number of sequences (from 1 to the cardinality of the sequences subset) overpass its threshold.

The Probe System reproduces the sequences within the sequences subset and reports the measure times for each one. The Application Surveillance System uses that reports to check threshold overpassing and triggers an action if the condition in bullet 5 above is met, among the following ones:

^■ Command a corrective action to the Scaling and Healing System , among the following ones (only for applications deployed and running on an internal and controlled IT infrastructure):

• Scale up the application, e.g. create a new instance of some of the components of the application

• Perform a corrective application healing action, e.g. reset some failing components in the application.

^■ Notify (email to a given user, creating an issue in a ticketing system, SMS, etc.) through the Notification System.

In the case that 6 were to make a corrective action, the Application Surveillance System checks that the success or failure of such action, checking the triggering condition again as specified in step 6 after a time T_w- If the triggering condition stills, then surveillance on that application is disabled (as per procedure E) and a notification (email to a given user, creating an issue in a ticketing system, SMS, etc.) is sent to the user through the Notification System.

Re-calibrate a recorded navigation sequence:

1. The Frontend checks database and offers a list of the navigation sequences recorded and already calibrated by that user (according to procedure B). The user chooses a sequence, which an associated <T_Gi, T_Ai>.

2. Steps 2 and 3 of procedure B "Calibrate navigation sequence" are performed. The new T_R time is stored in the database associated to the sequence, replacing the old one.

3. If the re-calibrated sequence belongs to the navigation sequence set of any surveillance (as defined in Procedure C), its U, is recalculated based on the new value of T_R,

Disable surveillance on application:

1. The Frontend offers a list of applications being surveyed by the user (according to procedure C). The user selects one of them.

2. The surveillance on that application is disabled, i.e. the surveillance configuration for the application is not removed, but the Probe Systems stops reproducing sequences on it and corrective/notification actions are not taken (steps 6-7 in procedure C).

Enable surveillance on application:

1. The Frontend offers a list of applications managed by that user which surveillance configured but disabled (according to procedure E). The user selects one of them.

2. Application surveillance is enabled, i.e. steps 6-7 in procedure C are performed again.

Remove surveillance on application:

1. The Frontend offers a list of applications being surveyed by the user, enabled or disabled (according to procedures C, E, F). The user selects one of them. 2. If application was enabled, it is disabled (according to procedure E). Surveillance configuration is removed (i.e. Tgju, and combined triggering conditions).

H Remove recorded navigation sequence:

1 . The Frontend checks database and offers a list of the navigation sequences recorded by that user (according to procedure A) that are not being used by any surveillance (according to procedure C). The user selects one of them.

2 The navigation sequence is deleted from database, including all its associated information.

Advantages of the Invention:

The invention includes advanced features such as autocalibration loop for navigation sequences, navigation sequence anomalies detection, quality limits recommendations, navigation sequences combination and flexible action triggering with success check.

A person skilled in the art could introduce changes and modifications in the embodiments described without departing from the scope of the invention as it is defined in the attached claims.

Claims

1. - A method to surveil applications of online services, comprising monitoring at least one web application in order to check and control operation of said at least one web application, said at least one web application deployed and running in an internal or external controlled Information Technology (IT) infrastructure and adapted to be accessed for at least one user device with computing capacity, said method being characterized in that it comprises acquiring, by means of a surveillance application including a frontend unit, a database and a recording unit, a plurality of navigation sequences representing the behavior of said at least one user with said at least one web application while providing an online service, said surveillance application interacting with at least one user device with computing capacity.

2. - A method according to claim 1 , characterized in that said surveillance application comprises recording by said recording unit each one of said plurality of navigation sequences.

3.- A method according to claim 2, characterized in that it comprises storing in said database said recorded navigation sequences.

4.- A method according to claim 3, characterized in that it further comprises storing an identifer of said at least one web application regarding in which context said navigation sequences have been recorded.

5.- A method according to any of previous claims, characterized in that it comprises providing communication by means of a plurality of interfaces about interactions of said surveillance application with a programmable probe system, a scaling and healing system and a notification system external to said surveillance application.

6. - A method according to claim 5, characterized in that it comprises calibrating said stored and recorded navigation sequences by choosing said at least one user device with computing capacity any sequence of said recorded navigation sequences.

7. - A method according to claim 6, characterized in that it further comprises passing said chose sequence to said programmable probe system that reproduces said sequence a plurality of times and takes a mean value of a reproduction time in order to perform said calibration.

8. - A method according to claim 1 , characterized in that it comprises detecting anomalies of said plurality of navigation sequences.

9. - A method according to claim 1 , characterized in that it comprises defining a quality limit for each one of said plurality of navigation sequences.

10. - A method according to claim 9, characterized in that it comprises defining said quality limit by using an absolute value of a perceived navigation time, a percentage (p) of said perceived navigation time, a degradation factor (u,) and/or a recommendation done by said application surveillance for either said u, or said percentage of said perceived navigation time.

1 1. - A method according to claim 1 , characterized in that it further comprises defining a triggering condition based on a combination of said plurality of sequences.

12. - A method according to claim 1 1 , characterized in that it comprises performing said triggering when at least one sequence of said combined sequences are above an effective time threshold (U,).

13. - A method according to claim 1 1 , characterized in that it comprises performing said triggering when all of said combined sequences are above an effective time threshold

(U;).

14. - A method according to claim 12 or 13, characterized in that said effective time threshold (U,) is defined using one of the following formula:

U = T ^{G ~ T} = T ^{G ~ T}

RT_G-T_T ^{~ R} T_A

T_r'-T_T

U = T_R l + u), u= ^—^-l

15. - A method according to claim 1 1 , characterized in that it further comprises checking the success or failure of said triggering.

16. - A system to surveil applications of online services, said system to surveil online services comprising:

said at least one web application deployed and arranged to run in an internal or external controlled Information Technology (IT) infrastructure and adapted to be accessed for said at least one user device with computing capacity; and

a plurality of computational, storage and network resources provided by said internal or external controlled IT infrastructure,

said system characterized in that it comprises a surveillance application adapted to control said at least one web application in order to acquire a plurality of navigation sequences representing the behavior of said at least one user with said web application while providing an online service.

17. - A system according to claim 16, characterized in that said surveillance application comprises: - a frontend unit arranged to provide a user interface to said at least one user device with computing capacity using said system;

- a recorder unit arranged to record said stored plurality of navigation sequences; and

- a plurality of interfaces arranged to establish communication with a probe system, a scaling and healing system and a notification system.

18.- A system according to claim 17, wherein said system is adapted to implement a method according to any of previous claims 1 to 15.