WO2017034512A1

WO2017034512A1 - Interactive analytics on time series

Info

Publication number: WO2017034512A1
Application number: PCT/US2015/046222
Authority: WO
Inventors: Renato Keshet; Gowtham Bellala; Dorin DANIAL; Reuth Vexler; Alina Maor
Original assignee: Hewlett Packard Enterprise Development LP
Current assignee: Hewlett Packard Enterprise Development LP
Priority date: 2015-08-21
Filing date: 2015-08-21
Publication date: 2017-03-02
Anticipated expiration: 2018-02-21

Abstract

Interactive analytics on a time series is disclosed. One example is a system including a monitoring module., a clustering module, an interaction module, a machine learning module, and a signal analysis module. The monitoring module receives a time series via a processor. The clustering module automatically segments the time series, and generates a plurality of clusters based on similar segments. The interaction module provides cluster representatives associated with the generated clusters to a computing device via an interactive graphical user interface, and identifies user interaction with at least one cluster representative. The clustering module modifies the at least one cluster representative based on the user interaction. The machine learning module learns from the user interaction to train a signal analysis module to automatically analyze another time series.

Description

INTERACTIVE ANALYTICS ON TIME SERIES DATA

Background

[0001] A variety of analytic tasks may be performed on data, and the results may be provided to a user. Analysis and interpretation of performance data from a device enables a user to diagnose and detect a faulty device.

Brief Description of the Drawings

[0002] Figure 1 is a functional block diagram illustrating one example of a system for interactive analytics on a time series.

[0003] Figure 2 is an illustration of an example interface for interactive analytics on a time series.

[0004] Figure 3 is an illustration of another example interface for interactive analytics on a time series.

[0005] Figure 4 is an illustration of an example interface for fault detection in a monitored system.

[0006] Figure 5 is a block diagram illustrating one example of a computer readable medium for interactive analytics on a time series.

[0007] Figure 6 is a flow diagram illustrating one example of a method for interactive analytics on a time series.

Detailed Description

[0008] A network of devices may be monitored to quickly detect anomalous behavior. For example, in an Internet-of-Things a fleet of devices may

be connected to a remote hub. Each device may be equipped with a plurality of sensors that measure performance and/or status of its particular sub-systems, and the hub may receive these signals (e.g., time series measurements) continuously, and optionally, in real-time. The hub may monitor the signals from the plurality of devices, interpret their meaning, and report the health of the devices on a dashboard, by sounding an alarm, or through any other messaging system.

[0009] In some situations, a hub may be configured to perform such operations. For example, an automatic system may be configured to recognize anomalies, rare events, strange behaviors, and so forth. For example, in a medical context, a system may monitor a patient's health based on an analysis of heartbeat measurement signals.

[0010] Existing expert systems are generally built as a result of projects involving professionals that may include a subject matter expert ("SME"), a data scientist, and an information technology ("IT") professional. In the course of the project, the SME may explain to the rest of the team how he/she reads signals, interprets their meaning, and recognizes relevant events. For example, the SME may be a cardiologist trained to read heartbeat measurement signals to monitor a patient's cardiac health. Alternatively, during the project, the SME may tag large quantities of signal data, which can be used as an indicator on how to read and interpret the signals. However, an SME is not trained in signal analysis or machine learning, and therefore he/she cannot produce an automated algorithm that reads and interprets signal data. A data scientist is trained in signal analysis or machine learning, and may incorporate the SME's knowledge and analysis into an algorithm. The IT professional may then encode the algorithm into an efficient, scalable, and robust code that may be implementable via a computing device and/or system. This code usually produces a report to output the analytics results. Alternatively, the code may send the results by some messaging system.

[0011] Generally, the SME, data scientist, and IT professional are not familiar with specifics of each other's application area. Accordingly, there is a need to bring them together for knowledge transfer to take place. Such a scenario is expensive and not scalable. For example, enormous financial and time resources may be spent to bring an SME and data scientist together to share and transfer relevant know-how. Moreover, after an initial system is created based on an analytics project involving an SME, a data scientist, and an IT professional, it may be difficult to continually modify the system, and/or redesign the reporting format of the analytics results of the system. Modifying the system or redesign of such a reporting and/or visualization tool may require a second analytics project involving additional SMEs, data scientists, and IT

professionals.

[0012] Accordingly, there is a need for a computer-implemented system that enables SMEs to explore their data and obtain actionable insights, without the need for expensive analytics projects. Interactive analytics enables SMEs to interactively explore their data and obtain relevant insights, without the need for a data scientist or an IT professional. Generally, an interactive data

visualization tool may be provided, including automatic event categorization and anomaly detection. The SME may interact with the data and provide

recommendations. Generally, the SME interacts with results of analytics that have already been generated by the system. For example, the system may generate anomalies, identify patterns, generate clusters, and so forth based on raw data, and /or labeled data, and the SME interacts with the anomalies, patterns, and/or clusters to provide subject matter insight. The system then learns from such user interaction to decide what is or is not relevant to the user. Based on a symbiotic relationship between the user and the system, the system may be trained to become an expert system directly from the expert.

Additionally, the SME need not spend an enormous amount of time transferring knowledge and insights. Instead, the system generates results of analytics, and the SME interacts with the system to provide feedback and steers the system to relevant insights.

[0013] As described herein, the SME interacts with the system after the system has generated initial results of data analytics performed on incoming data. In some instances, parameters for performing the initial data analytics may be provided by the SME, or may be based on a previously analytics trained model. When the system provides the SME with the results, the SME may further tag the results and/or direct the system to modify the results. Accordingly, the system may be trained to intelligently pre-process additional data.

[0014] As described in various examples herein, interactive analytics on a time series is disclosed. One example is a system including a monitoring module, a clustering module, an interaction module, a machine learning module, and a signal analysis module. The monitoring module receives a time series via a processor. The clustering module automatically segments the time series, and generates a plurality of clusters based on similar segments. The interaction module provides cluster representatives associated with the generated clusters to a computing device via an interactive graphical user interface, and identifies user interaction with at least one cluster representative. The clustering module modifies the at least one cluster representative based on the user interaction. The machine learning module learns from the user interaction to train a signal analysis module to automatically analyze another time series.

[0015] In the following detailed description, reference is made to the

accompanying drawings which form a part hereof, and in which is shown by way of illustration specific examples in which the disclosure may be practiced. It is to be understood that other examples may be utilized, and structural or logical changes may be made without departing from the scope of the present disclosure. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims. It is to be understood that features of the various examples described herein may be combined, in part or whole, with each other, unless specifically noted otherwise.

[0016] Figure 1 is a functional block diagram illustrating one example of a system 100 for interactive analytics on a time series. System 100 includes a monitoring module 104, a clustering module 106, an interaction module 108, a machine learning module 110, and a signal analysis module 112. In some examples, the monitoring module 104 receives a time series 102 via a processor. Time series 102 may be generally representative of any data received via a processor. In some examples, time series 102 may be

performance data from a monitored system. The term, "monitored system" as used herein may generally refer to any monitoring system that gathers data. In some examples, such data may be real-time data. In some examples, the monitored system may include a person. For example, a person may be undergoing medical diagnostics, and the person may be communicatively linked to provide medical data. In some examples, the monitored system may be a monitored device or a monitored equipment. In some examples, the monitored system may be a communicatively linked network of monitored devices and/or equipment. For example, a monitored system may include a plurality of HVAC devices communicatively linked to provide performance data for a sub-plurality of the HVAC devices.

[0017] In some examples, the time series 102 may be an input in the form of a signal, such as, for example, sensor readings, performance metrics, or any time series from the monitored system. In some examples, the time series 102 may be an input in the form of a plurality of signals. In some examples, system 100 may include a conversion module (not shown in the figures) to convert performance data from a monitored system into the time series 102.

[0018] More formally, there may be S input signals, where each signal may be a list of numbers, [s_j (n)}, called samples, where

S-1. Each sample, {s, (n)}, may be associated with a time interval,

nAt, where t₀ is the time of the first sample, and At is a fixed time interval.

[0019] System 100 includes a clustering module 106 to automatically segment the time series 102, and generate a plurality of clusters based on similar segments. In some examples, the time series 102 may be divided into temporal windows. For example, the collection of samples may be divided into

(overlapping or non-overlapping) time windows of length L. In some examples, the window length L may be specified by the user.

[0020] For ease of description, the windows may be assumed to be non- overlapping. In some example, each non-overlapping window,

.... may be represented as a tabular array:

[0021] In some examples, each window u_t may be mapped to a d-dimensional feature space F. In other words, the window representation for u_it may be provided as:

[0022] In some examples, the mapping M may be:

[0023] In some examples, initial clustering on a subset of the windows may be performed by the clustering module 106. The subset may be randomly chosen from the list of mapped windows, and its size may be X% of the total number of windows. The goal of this initial clustering is to obtain key initial parameters (e.g., cluster variances) to be used in the main clustering routine, described herein. In some examples, the initial clustering may be a variant of K-Means applied to the window subset. The input data may be normalized, such as, for example, based on a normalized Euclidean distance applied to the feature space, F, as a distance function utilized by the K-Means procedure:

where af is a variance of feature I among all windows in the subset. [0024] In some examples, the value of K in K-Means procedure may be determined based on a technique of clustering indices. Generally, a value for K may be predefined from a very large range by minimizing a cost function based on the relationship between inner-cluster densities and inter-cluster distances. In some examples, clustering indices may include at least one of Ball-Hall indices, C- indices, and Calinski- Harabasz indices. Indices and appropriate cost functions may be described more formally. Let W denote a matrix constructed of n row- vectors w_h where each row-vector has length LS. In some examples, the data may be partitioned into K clusters,

denote centroids of the clusters and |Q| the cardinality of each

cluster. Furthermore, we may let G denote a centroid of the entire dataset W, and let WGSS¹ denote a within-cluster dispersion, i.e., a sum of distances between the cluster centroid and the rows of W belonging to the /-the cluster:

[0025] In some examples, the Ball-Hall index may be a mean, through all the clusters, of their mean dispersion, i.e., the C_ind index may be defined as:

[0026] Accordingly, an optimal / may be one that gives the smallest C_ind. The advantage of this index is in its simplicity of implementation. While it may seem that the larger the value of K, the smaller the index (in the limit providing one- element clusters) as WGSS¹ gets smaller; however, as the size of the cluster \C_t \ is in the delimiter, it serves as a contra-weight, bringing the C_ind to a certain compromise between the density of the clusters and their number.

[0027] In some examples, clustering index may be a C-index. Generally, the C- index is aimed at balancing between inter and intra cluster dispersions. Distances between pairs of points inside each cluster may be considered. Let N_w define the total number of distinct pairs of vectors (pairs of points) in all the clusters, summed separately among the different clusters. Let N_T define the total number of pairs of distinct points in the entire data set, i.e., the number of distinct pairs between the vectors of matrix W. A sum of the N_w distances between all the pairs of points inside each cluster may be determined, and denoted as S_w. A sum of N_w smallest distances between all the pairs of points in W may be determined and denoted as 5_min. Generally, there are N_T pairs of which N_w smallest distances are selected. Likewise, a sum of N_w largest distances between all the pairs of points in W may be determined and denoted as S_max. Accordingly, the C-index, C_lnd, may be determined as:

[0028] In some examples, an optimal / may be determined based on a maximum difference between successive slopes. Let {fcj denote the range of values evaluated as candidates for optimal ity, and let associated C_ind values be denoted as

Then, an optimal / may be determined based on a series of derivatives

[0029] In some examples, clustering index may be a Calinski-Harabasz index. Generally, this index is a good criterion for clustering when the groups of clusters are of approximately the same size. Let the pooled within-cluster sum of WGSS be the sum of within cluster dispersions among all clusters, determined as:

[0030] Similarly, an inter-cluster dispersion BGSS may be defined. Generally, such a dispersion may be equivalent to a dispersion of centroids of the clusters with respect to the centroid of the entire dataset, and may be determined as:

[0031] Accordingly, the Calinski-Harabasz index may be defined as:

In some examples, it may be desirable for clusters to be as disperse as possible, and therefore, an optimal / may be one that maximizes C_ind.

[0032] In some examples, system 100 may include a classifier module (not shown in the figures) to associate the generated clusters with binary classifiers. For example, after the initial clustering, K classifiers may be trained, one for each cluster. For each cluster c, a binary classifier p_c may be generated. The classifier p_c returns the probability p_c (u_f) that a window u_t belongs to the cluster c. The cluster c may be either one generated in the initial K-Means clustering described above, or a new cluster obtained in a main clustering routine as described herein. In some examples, a maximum-likelihood version of a

Gaussian Naive Bayes Classifier may be utilized. In a cluster c, each feature, for example, as defined in (Eqn. 2), may be assumed to have a Gaussian distribution of mean μ_ε (/) and standard deviation a_c (/). The features may be assumed to be independent of each other, and accordingly, the classifier p_c may have the following format:

where

(Eqn. 8)

[0033] In some examples, the classifier p_c may also be expressed as:

obis distance between the

1

[0034] In some examples, only the Mahalanobis distances, , may be

calculated instead of the actual probabilities p_c(u_{). This is because nominal probability values may not be needed, but only their relative intensities may be needed, and computing distances may be sufficient since

if and only if D Window features, as described herein, are

means and variances. Accordingly, it may be noted that \i_c (Z) are means of means and variances, and a_c (Z) are standard deviations of means and variances.

[0035] In some examples, the monitoring module 104 may receive another time series via the processor. In some examples, the another time series may be a portion of time series 102. In some examples, the another time series may be a time series different from time series 102. For example, time series 102 may be received from a first monitored system, whereas the another time series may be received from a second monitored system.

[0036] In some examples, the clustering module 106 may automatically segment the another time series, and the signal analysis module 112 may automatically analyze the another time series by associating each segment of the another time series with a known cluster based on an associated binary classifier. For example, once clusters are associated with respective binary classifiers, system 100 may associate new segments with existing clusters based on these binary classifiers. In some examples, each window ¾ may be checked against each of the existing classifiers p_c. In some examples, a cluster c_t may be selected so that is a maximal, i.e., In some examples, this may

be equivalent to

where

is the Mahalanobis distance as in Eqn. 10.

[0037] In some examples, segments (or windows) may be associated with clusters based on the Mahalanobis distances. For example, when the distance of a window / to its closest cluster £_t exceeds a threshold 7, the signal

may be considered to be an anomaly, and a new cluster may be defined;

otherwise, the window / may be associated with the cluster d₀ In some examples, the threshold T may be input and/or modified by a user via the computing device 114. When a window is determined to be an anomaly, and a new cluster c_n is generated, its associated binary classifier may also be a Gaussian Naive Bayes Classifier, with a mean {μ_ίη(θ} and a standard deviation

- Such a new mean may be set to be identical to the features of the anomalous

Likewise, the standard deviation of the new classifier may be set to be identical to that of the closest cluster, i.e.

[0038] Generally, the Mahalanobis distance between a sample and a d- dimensional Gaussian distribution follows a Chi-Squared distribution with d degrees of freedom. Accordingly, for a given confidence level p, a window / may be considered to be an outlier and/or anomaly with respect to a cluster c when

where EJ¹ is an inverse of the cumulative Chi-Square distribution with d degrees of freedom.

[0039] Generally, the parameter p may be chosen to be close to 1. Some typical values may be 0.9, 0.99, or 0.999. In some examples, this parameter may be modified by the user in order to control the number of clusters obtained by the clustering routine. The closer p is to 1 , the larger ¹ (p) is, and the smaller the number of clusters are. To increase the number of clusters, the parameter p may be reduced, and in some examples, it may be chosen to be significantly smaller than 1.

[0040] In some examples, the parameter p may be modified to a resolution parameter, denoted by r, indicative of how strict a similarity between the cluster elements may be. One example relationship between rand p may be

[0041] Generally, Vr ^may control the number of "nines" after the decimal point in p. As the resolution r decreases, the confidence level p approaches 1 , which may cause less windows to be considered outliers, or, as a consequence, less new clusters may be created. When r tends to infinity, p tends to zero; when p = 0 all elements in a cluster have to be identical in the feature space, so every new window may create a cluster of its own. Initially, as system 100 is run for the first time, the value of the resolution is r = 1. This value may be replaced (e.g., by a user) by any positive real number.

[0042] By utilizing Eqn. 15 in Eqn. 14, the following threshold may be obtained:

which may be utilized to determine if a window and/or segment is anomalous with respect to a cluster c, based on the threshold T:

[0043] In some examples, for values of rthat are in the interval [0.1, 10], the threshold T may be approximated as:

which may simplify implementation. Table 1 is a list of values of

( ) for

[0044] System 100 includes an interaction module 108 to provide cluster representatives associated with the generated clusters to a computing device 114 via an interactive graphical user interface 116, and to identify user interaction with at least one cluster representative. The term "cluster

representative" as used herein, generally refers to a graphical rendition of a cluster for display via the interactive graphical user interface 116. In general, a user interaction with a cluster representative is identified by the interaction module 108, which in turn prompts the clustering module 106 to modify a cluster associated with the cluster representative.

[0045] In some examples, the monitoring module 104 may receive a collection of signals. As described herein, the input may comprise signals, which may be sensor readings, performance metrics, or any time series. The monitoring module 104 may be generally capable of receiving multi-signal input as well as a single signal input. Accordingly, the output of system 100 may be a list of machine-learning models, each characterizing a different signal-event cluster, as well as a list of actions associated to each cluster. The clustering module 106 may receive the input from the monitoring module 104, and may segment each signal of the collection of signals into temporal windows. In some examples, the clustering module 106 may check each window against an existing labeled cluster to identify a potential match. In some examples, the window may match an existing cluster, and the clustering module 106 may associate the window with the identified cluster. In some examples, windows that are not associated with an identified cluster may be clustered into a new (e.g., unlabeled) cluster. [0046] In some examples, the interaction module 108 may provide the labeled and/or unlabeled clusters to the computing device 114. A user may visualize the signals and the clusters on the computing device 114. In some examples, the user may zoom into portions of the signal to examine the signal in more detail.

[0047] In some examples, the interaction module 108 may provide the clusters represented in the form of rectangles (as illustrated in Figure 2). In some examples, the interaction module 108 may provide the clusters based on multidimensional scaling ("MDS"). In an MDS representation, clusters are represented as circles, where a size of the circle may be based on a number of elements in the cluster represented by the circle. In some examples, a distance between two clusters may represent a relative similarity between the two clusters, where circles representative of more similar clusters may be provided in closer proximity to one another than for circles representative of less similar clusters. Accordingly, an MDS representation is a graphical representation of sizes and relative similarities of clusters, and may allow the user to visually group the clusters.

[0048] Generally, clustering may be based on a particular application. For example, consider a test case where the time series is indicative of performance data from a monitored system, where the monitored system includes air- conditioning systems. In such an instance, the time series may represent an operation of different cooling/heating/fanning devices as well as reading sensors' data collected from different sensors in a system, and so forth. In such a system clusters may be based on various system operational modes, e.g., cooling, heating, ventilation with external sources of air (outside air),

dropping/raising the temperature while some event occurred in the system. Generally, system 100 may be trained to provide a rationalization behind each of the clusters, as well as provide the user the option to focus on significant signals that may effectively represent the entire dataset. In some examples, sorting and ranking of an entropy of different signals /(. ), either among the entire representative data-set or among the representative signals within each of the clusters, may be utilized.

[0049] For example, in the matrix W, constant features (or column vectors) may be eliminated as being the same in different clusters, and therefore as being non-interesting in distinguishing between clusters. Entropies may be

determined for each non-constant feature. Generally, the lower the entropy of a feature, the less the feature changes among all data-points belonging in the cluster, and therefore, the feature is more representative of the cluster, i.e., provides its rationalization. In some examples, the clustering module 106 may identify a certain number, j, of such features and present their statistics to the user. The number, may be predefined by either the user, or may be automatically set by system 100.

[0050] Such a process may be applied to the entire dataset to obtain entropies of non-constant features. Common features among clusters are generally those with lowest non-zero entropies, or the features that change the most (e.g., those that define the variability of the dataset). In some examples, the user interaction provides such feedback to the machine learning module 110 to automatically cluster the segments. [0051] In some examples, the clustering module 106 may segment the incoming time series 102 into windows {wj of length L that may provide the highest "clustering accuracy". Generally, windows that might have maximum

resemblance may be identified based on a certain similarity metric. For example, consider an Euclidean similarity metric, such as an I² norm, where a system may be trained to distinguish shapes (edges, etc.). In such a case, clustering may be ambivalent as to a precise location of the shape as long it exists in the window, and it is in a certain range of values (small edges, medium jumps, abrupt changes, and so forth).

[0052] In some examples, two windows may contain an edge of the same form. In situations where windows are created as non-overlapping windows, the I² norm may associate two such windows with different clusters. On the other hand, when one of the windows is slightly moved so that the edges of the windows may be positioned at approximately the same location, the clustering results may be of much higher accuracy. Generally, non-overlapping windows may be used, but in some examples, a window may be allowed to move and overlap with the preceding/successive window up to half of the window size, i.e., L/2. More precisely, a window that contains the highest energy among the shifted windows may be selected. More formally, the energy for each window sliding position may be computed

Generally, Eqn. 22 represents a mean of the energy of the signals, when the signal means are first subtracted. Accordingly, the window sliding position may be selected as the one that is associated with a maximum energy p_t.

[0053] In some examples, the clustering module 106 may distinguish between a pair of clusters based on mutual information content between the pair of clusters. More formally, assume there are two random variables X and Y with marginal densities and respectively. The joint probability density

between X and Vis defined by

The mutual information is then given by:

[0054] In some examples, system 100 may not have explicit access to

probability densities. Instead, probability densities may have to be estimated based on a Gaussian kernel estimator from samples, such as, rows of matrix W. For example, the clustering module 106 may determine the probability densities as follows:

[0055] The probability density f may be defined similarly based on Eqn. 25.

Here, h is a tuning parameter that defines the width of the Gaussian kernel and may, in some examples, be chosen to be 1/10 of each feature variability (tenth of the standard deviation for each of the features). In theory, this parameter may be provided externally to the computing device, and a user interaction may select and/or modify h manually to another percentage of the variability or even to a constant number for all the features. Hence, the entropy of each random variable may be given, based on an estimated probability density from Eqn. 25, by:

[0056] In some examples, the clustering module 106 may identify cluster features to distinguish between clusters. As described herein, distances between MDS clusters may be utilized to distinguish between clusters, and identify pairs of closest clusters. Also, for example, a largest distance between two clusters may be utilized to distinguish between clusters. In some examples, mutual information may be utilized to distinguish between clusters. For example, the clustering module 106 may take each such two clusters and calculate the entropy of the clusters features choosing features with the highest entropy as most distinguishing between the two clusters. Also, for example, the clustering module 106 may identify a subset of features that have been selected (e.g., via user interaction) to represent each cluster, and determine a largest mutual information among this subgroup of identified features. Generally, the smaller the mutual information, the more different the behavior of this

representative feature between the two clusters under comparison. Such an information theory based comparison allows for comparison of any two clusters, instead of closest (or similar) clusters.

[0057] In some examples, the mutual information content between the pair of clusters may be based on a Kullback-Leibler Divergence. Generally, a higher divergence between two clusters over the same feature, indicates that the feature is more distinctive. Other (informational-theoretic and other) measures of similarity may be applicable as well, based on a variety of use-cases.

[0058] In some examples, the data size may be large, leading to a time- consuming determination of entropies. Accordingly, entropy calculation may be limited to a suitable subset of the dataset, for example, a predetermined number, say 2000 vectors. In some examples, each cluster may be randomly sub-sampled according to relative cluster size with respect to the size of the dataset.

[0059] In some examples, the interaction module 108 may identify user interaction related to analysis of at least one cluster. For example, the user interaction may include dragging a first visual representation of a first cluster to a second visual representation of a second cluster to merge two clusters. In some examples, the clustering module 106 may modify the at least one cluster based on the user interaction. For example, the interaction module 108 may identify the user interaction and prompt the clustering module 106 to merge the first cluster and the second cluster. More formally, let _t and

be two clusters

that are to be merged. Let _t and N be the number of elements assigned to

clusters c_v and respectively. Let c denote the merged cluster. A mean

M fo^r the merged cluster may be determined based on a weighted

average of means

and

of clusters

and where the weights may be

based on relative cluster sizes

and In some examples, the mean for the

merged cluster

may be determined as:

[0060] Likewise, a standard deviation for the merged cluster

_m may be

determined based on a weighted average of standard deviations a and

for clusters _t and respectively as:

[0061] As another example, the user interaction may include splitting a cluster representation of the first cluster to form at least another new cluster.

Accordingly, the interaction module 108 may identify such user interaction and prompt the clustering module 106 to split the cluster. In some examples, splitting a cluster may be implemented by applying K-means to the elements of the cluster, with K = 2. The means and standard deviations of the resulting two clusters may be computed as described herein.

[0062] Also, for example, the user interaction may include moving a data element from a first cluster representation of the first cluster to the second cluster representation of a second cluster. Generally, the first cluster is not changed. In some examples, the second (or destination) cluster may be a new cluster, and the clustering module 106 may generate a new cluster using the mean of the signals as its centroid. Feature statistic parameters may be determined based on a transform between the centroid and itself, and the output may be used as the mean features. In some examples, a standard deviation of 0.1 may be used for all features, where the value 0.1 may be chosen experimentally.

[0063] In some examples, the second (or destination) cluster may be an existing cluster, and the clustering module 106 may keep adding the moved signals in a loop to the second cluster, until the probability of belonging to the desired second cluster is the highest between all the other clusters. In some examples, the user interaction may include moving an element out of a cluster

representation of a given cluster to form a new cluster.

[0064] In some examples, the user interaction may include labeling the at least one cluster representation. For example, the user interaction may include validating a cluster (by validating the associated cluster representation), and associating the cluster with a name. As described herein, in some examples, a classification rule may be associated with a labeled cluster, where the rule determines membership of a window in the labeled cluster. In some examples, the user interaction may include associating the at least one cluster

representation with an alert. For example, an action may be associated with a labeled cluster representation.

[0065] Generally, prior to training a model based on labeled training data, system 100 may be prompted to determine the clustering based on incoming a time series. For example, a user may click on a "Run Signal" mode provided via the interactive graphical user interface 116, and the clustering module 106 may cluster the incoming time series. At the first time this request is called after system 100 is started, a 'configuration file' may be loaded and the data stream may be read from a csv file (e.g., as defined in the 'configuration file'). At this point, there is no trained model from previous runs, so system 100 generally runs without any predefined learned model. At other times when this request is called, an additional time series may be loaded (by searching for a file named exactly as in the configuration file but with the addition of a "new" suffix which indicates the next time series) and system 100 may search for a trained model that might be saved previously in the file system. Such a trained model may be loaded and utilized in the clustering algorithm.

[0066] In some examples, a "Restart" mode may be provided that generally functions similar to the "Run Signal" mode. The "Restart" mode may be useful in situations where a user may want to start the training process from the beginning without any trained model, or in cases the user may want to change some parameter in the configuration file and run the application with this change.

[0067] In some examples, the interactive graphical user interface 116 may provide at least one of a representation of data, a representation of the at least one cluster representative, at least one operations tab, and a cluster analysis window. In some examples, the representation of data generally includes information related to the original data. Generally, this may include the signal, features list, and a time scale text box.

[0068] Figure 2 is an illustration of an example interface 200 for interactive analytics on a time series. For example, Figure 2 shows labels provided by a domain expert to the most frequent clusters, where four major normal operating modes corresponding to OFF", "Heating", "Cooling with outdoor air", and "Mechanical cooling with 100% outdoor air" states are identified. In addition, three clusters corresponding to faulty or inefficient operating modes are also identified, with two clusters corresponding to "Inefficient economization mode" and "Inefficient heating mode" categorized as medium risk, and one cluster corresponding to "Supply fan turned ON during non-working hours" categorized as high risk. [0069] Some components of the example interface 200 may include a signal (e.g. a visual representation of the incoming time series) represented as graph 202 - this may display the entire data stream, the x-axis 204 corresponding to time and the y-axis 206 corresponding to the values. Some example user interactions may include hovering over a point in the graph 202 to show the type of each point, the coordinates and the cluster name of the cluster to which it belongs. In some examples, the user interactions may include double clicking on a point in the graph 202 to open a window "Cluster analysis" of the specific cluster of this point with a view of the samples including the point. In some examples, the user interactions may include right clicking and dragging a square to open a menu to enable zoom-in operations to this square. In some

examples, the user interactions may include clicking on a 'zoom out' label (for example, at the right top corner of the graph 202) to return to the original zoom of the plot. In some examples, the user interactions may include clicking on the arrows (e.g., at a right top comer of the graph 202) to enable navigation inside the plot. In some examples, the user interactions may include mouse scrolling to enable to zoom-in/zoom-out operations.

[0070] Some components of the example interface 200 may include a feature list 208. This feature list 208 may include feature names as written in a configuration file columns header. In some examples, the most significant features may be shown. Some example user interactions may include checking and/or unchecking items in the list to display and/or hide these items from the plot. Selectable items in the feature list 208 may include supply air temperature, cooling coil value, heating coil value, mass air, replacement, mixed air temperature, outside air temperature, and so forth. A time scale text box 210 may include the current time scale which the clusters are computed with. For example, if the time scale is set to 10 minutes, it means that the data was segmented into samples of 10 minutes, and each sample was classified to one of the clusters. This parameter may be editable only in a first run. Some example user interactions may include changing the 'Time scale' value, where the change may be effective only after clicking on a 'Refresh' icon. [0071] In some examples, the at least one cluster representative may be provided with information regarding the generated clusters, and display types to provide the at least one cluster representative (e.g., a rectangular display type as illustrated in Figure 2, an MDS display type as illustrated in Figure 3, etc.). Generally, the information regarding the generated clusters may include a clusters toolbar 212, as described herein. In some examples, the clusters toolbar 212 may include display options, and may include a selectable tab 'Sort by' to provide an option to order the cluster representatives by frequency (e.g., number of samples in each cluster), and discrepancy (e.g., the distance of a cluster's centroid from its most closest cluster). In some examples, the clusters toolbar 212 may include a selectable tab 'Represent cluster by' may include selectable tabs to represent the generated clusters by an average. For example, a histogram with number of checked feature columns may be provided, with each column representing an average value of this feature in all samples that are included in this cluster (the value may be normalized to a number between 0 and 1 ). Also, for example, a graph representing the centroid of this cluster may be provided, with the x-axis corresponding to time and the y-axis corresponding to the values of the features. In some examples, the clusters toolbar 212 may include selectable tabs to represent the generated clusters by an MDS representation, with each circle corresponding to a cluster centroid and the distance between clusters corresponding to the MDS distance between the centroids. The radius of each circle may be determined by the frequency/discrepancy of the cluster that may depend, in some examples, on the 'Sort by' value.

[0072] In some examples, the clusters toolbar 212 may include selectable tabs to represent the generated clusters by a 'Display filter¹ that may include a selectable tab for "show all" to show all the clusters representatives; a selectable tab for "labeled" to show only representative of labeled clusters; and a selectable tab for "unlabeled" to show only representative of clusters that are not associated with a label. In some examples, the clusters toolbar 212 may include selectable tabs to represent the generated clusters by a 'Resolution' 214 that controls the sensibility level of similarity inside a given cluster, where a greater value defines a less strict threshold for similarity and thus leads to creation of fewer clusters because more samples are classified as belonging to existing clusters. In some examples, the changes in clusters after an edit of this value may be effective after the 'Refresh' button is clicked.

[0073] In some examples, the at least one cluster representative may be provided with additional and/or alternative information regarding the generated clusters. For example, the at least one cluster representative may be visualized according to 'Represent cluster by' value. For example, a given cluster may be illustrated as a cluster representative associated with an "Off mode 216, a cluster representative associated with an "Inefficient Heating" mode 218, a cluster representative associated with a "Heating" mode 220, a cluster representative associated with a "Cooling with Outside Air" mode 222, a cluster representative associated with an "Inefficient Economization" mode 224, a cluster representative associated with a "Supply Fan Left On" mode 226, and so forth. Some example user interactions may include hovering over a cluster representative to highlight, on the graph, points which belong to this cluster; double clicking on a cluster representative to open a 'Cluster

investigation' window for the specific cluster, and dragging and dropping one cluster representative onto another in order to merge clusters.

[0074] In some examples, the at least one operations tab may generally include a "run signal" mode 228, a "refresh" mode 230, a "restart" mode 232, and a "save changes" mode 234. In some examples, clicking on the 'Run signal' 228 mode without a clicking to 'Save changes' 234 may present cluster

representatives for future signals without a learning process from the current interactions.

[0075] In some examples, the at least one operations tab may generally include the "Refresh" Mode 230 described herein. The "Refresh" Mode 230 functions similarly to the 'run signal' mode 228, but without taking into effect the user interactions since the last time the 'Save changes' 234 was clicked. In some examples, this request for a "Refresh" Mode 230 may take as input, a

'Resolution' and a Time scale' that the user may insert in a designated text box (a default setting may be the original parameters which came from the first run of the system). This request enables testing of different parameters such as 'Resolution' and Time scale' without any change to the model. In some examples, the at least one operations tab may include the "Restart Mode" 232 as described herein.

[0076] In some examples, the at least one operations tab may include the "Save changes" Mode 234 that generally saves all the user interactions and uploads them to a backend processing system, to be applied to the training process, and whereby the model is modified accordingly. For example, the current session may come end, and the next time the 'Run signal' mode 228 is clicked, the next stream of data with the updated and trained model may be analyzed and displayed.

[0077] In some examples, the "cluster analysis window" may generally enable analysis of a given cluster, including an analysis based on samples, properties, and statistics. In some examples, the "cluster analysis window" may be opened by double clicking on cluster's representative or by double clicking on a point in the graph. In some examples, the tabs for samples, properties, and statistics may include, for example a samples tab to provide samples that belong to a cluster. Some example user interactions may include clicking on a 'Split' button to prompt the backend processing system to split the cluster to two sub-clusters, and present the new suggested cluster representatives to the user. In this stage the user may confirm this suggestion by clicking on the "Save changes" Mode 234, or not confirm by clicking on 'back'. In some examples, the user interactions may include clicking on 'prev' (for previous) and 'next', in order to navigate between the samples. In some examples, the user interactions may include moving a sample from a first cluster to a second cluster, by dragging and dropping the first cluster inside the second cluster's representative div in the main window. In some examples, the user interactions may include moving a sample from a first cluster to a second cluster, by dragging and dropping it inside a 'New Cluster' div in the main window.

[0078] In some examples, the at least one operations tab may generally include a "Properties" Tab that shows the properties of the cluster (if any property was defined by the user). By defining properties for a cluster, the user directs the system to consider this cluster in the training process. Some example user interactions may include naming (or labeling) a cluster, and defining a cost, which may range from 'none' to 'high'. The cost may generally define a background color of this cluster in the graph. In some examples, the at least one operations tab may generally include a "Statistics" Mode to enable the user to see what the dominant features of a cluster are and also what the most similar cluster is.

[0079] Figure 3 is an illustration of another example interface 300 for interactive analytics on a time series. Some aspects of the example interface 300 of Figure 3 may be in common with aspects of the example interface 200 of Figure 2. The at least one cluster may be provided via an MDS display. As described herein, an MDS display provides cluster representatives in the form of circles. The incoming signal may be represented as a signal graph 302, with the x-axis 304 corresponding to time intervals (e.g., dates) and the y-axis 306

corresponding to the values (e.g., in the range 50 to 80).

[0080] A feature list 308 may list feature names that may be selected by a user. Some example selected feature names (indicated in the figure by a presence of an "X" next to the name) may include supply air temperature, cooling coil value, mixed air temperature, and outside air temperature. Some example feature names that are not selected by the user (indicated in the figure by an absence of an "X" next to the name) may include heating coil valve, mass air

replacement, return air temperature, and so forth.

[0081] In some examples, a time scale text box 310 may indicate a current time scale utilized to determine the clusters. As indicated in the figure, the time scale may be set by the user, and in this example, the time scale text box 310 is in hours, and may be set to a value of "3". This means that the data was segmented into samples of 3 hours, and each sample was classified to one of the clusters. As described herein, this parameter may be editable only in a first run.

[0082] The clusters toolbar 312 may include display options (as described with reference to Figure 2). For example, an option for "Sort by" may be selected to "frequency"; an option for "Represent cluster by" may be selected to "scatter"; and an option for a "Display filter" may be selected to "show all". Also, an option for "Resolution" 314 may be set to a value of "8.5" to control the sensibility level of similarity inside clusters.

[0083] As illustrated, the clusters are provided in an MDS representation, where each circle corresponds to a cluster centroid and the distance between clusters corresponds to an MDS distance between the centroids. The radius of each circle may be determined by a frequency/discrepancy of the cluster. Distances may be represented via a horizontal and vertical scale. For example, the horizontal axis 318 may be determined based on a range of values from -1.2 to 1.0, with an increment of 0.2. Likewise, the vertical axis 316 may be determined based on a range of values from -1.0 to 0.5, with an increment of 0.5. An example legend 320 is illustrated to represent the clusters. In some examples, a user may label a cluster as "high temperature", "low activity", and so forth. In some examples, such labels may be automatically assigned by a signal analysis module (e.g., signal analysis module 112 of system 100 of Figure 1 ). In some examples, clusters may be color coded, and legend 320 may represent the cluster name with its color label.

[0084] As illustrated herein, some cluster representatives are illustrated as a cluster associated with a "high temperature" mode 322, a cluster associated with a "low activity" mode 324, an unlabeled cluster "cluster 3" 326, and another unlabeled cluster "cluster 4" 328. Some example user interactions may include hovering on cluster's representative to highlight on the graph the points which belong to this cluster, double clicking on cluster's representative to open a window 'Cluster investigation' for the specific cluster, and dragging and dropping one cluster's representative to other in order to merge clusters. In some examples, the at least one operations tab may generally include a run signal mode 330, a refresh mode 332, and a save changes mode 334.

[0085] Referring again to Figure 1 , in some examples, system 100 may include a machine learning module 110 to learn from the user interaction to train a signal analysis module 112 to automatically analyze another time series. In some examples, the another time series may be indicative of performance data from a monitored system, and the trained signal analysis module 112 may automatically analyze the another time series for fault detection in the monitored system.

[0086] For example, modern buildings are generally large, complex dynamic systems that consume significant amounts of resources such as electricity, water and gas. The United States Energy Information Administration estimates commercial buildings in the U.S. to consume about 37% of the total energy generated (in other words, spend about $132 billion in annual energy costs alone). Moreover, it is estimated that 40% of these buildings have either misconfigured devices, and/or equipment that is poorly maintained and inefficiently operated, resulting in about 30 - 40% wastage in energy.

[0087] Many commercial buildings are equipped with Building Management Systems ("BMS") that monitor and control physical equipment such as air handlers, chillers, cooling towers, etc. These traditional BMS systems provide fault detection features that rely on simple rules created by operators (such as threshold or out-of-range check), thereby lacking the capability to identify equipment deterioration or inefficient operation leading to waste in energy.

Moreover, BMS systems are generally function driven and may not be energy efficient. This means that once a building is constructed and commissioned, the settings in the BMS systems are seldom changed as long as the whole building can still provide acceptable functionality, even though the energy performance may degrade, thereby resulting in increased, and generally undesirable, energy consumption.

[0088] In some examples, the trained signal analysis module 112 may automatically detect at least one of anomalies and rare events in incoming performance data based on the labeled training data. For example, the operation of a monitored system may be characterized by identifying the various operating states or modes. In some examples, inefficient or faulty operating modes that lead to wastage in building operations (such as HVAC equipment that is simultaneously heating and cooling a given space due to a failed sensor or other fault; or default configurations for all systems and pieces of equipment resulting in suboptimal set points) may be identified on a continuous basis. In some examples, various physical equipment in a building may be commissioned on a continuous basis, and benchmarked among its peers based on historical performance, thereby leading to timely identification of equipment that may need maintenance, and enabling predictive maintenance and control of equipment.

[0089] For example, signal analysis module 112 may be utilized for fault detection in Air handling units ("AHU"). An AHU may be used to regulate and circulate air as part of the heating, ventilation, and air conditioning (HVAC) system. A building typically consists of several AHUs. For example, a group of three buildings may include each building equipped with 4-8 AHUs, and each AHU may be equipped with 20-40 sensors. Each sensor may monitor parameters such as, for example, pressure,

temperature, flow rate, and power. These parameters may be accessed through a site BMS, which may be collected over a period of 6 weeks, and sampled every 5 minutes.

[0090] In some examples, historical a time series over 4 weeks may be utilized by system 100 to identify clusters associated with various operating modes of the AHUs. Based on user interaction with a domain expert, the machine learning module 110 may leam to identify normal vs.

faulty/inefficient modes.

[0091] Figure 4 is an illustration of an example interface 400 for fault detection in a monitored system. Some aspects of the example interface 400 of Figure 4 may be in common with aspects of the example interface 300 of Figure 3, and/or the example interface 200 of Figure 2. Performance data from a plurality of devices may be represented as a time series 402. The incoming signal may be represented as a signal graph 402 (comprising a plurality of individual graphs), with the x-axis 404 corresponding to time intervals (e.g., dates in November) and the y-axis 406 corresponding to the values (e.g., in the range 0 to 100).

[0092] The example interface 400 may include a first selectable menu 408 to toggle between a "Dashboard" view, a "Fault Detection" view, and an "Analytics" view. As indicated, the "Fault Detection" view is shown as the selected option. Also, for example, a second selectable menu 410 may enable the user to select from different types of devices, including "Automotive", and "Building".

Selections for "Settings" and "profile" may be included in the second selectable menu 410. Based on the user interactions, the system may be set to provide "Fault Detection" in "Building" devices.

[0093] In some examples, the plurality of devices may be HVACs in the

"Building" that are communicatively linked to AHUs. A legend 412 lists the HVACs (e.g., HVAC 1 , .... HVAC 9). In some examples, each HVAC may be associated with a color, and the time series representation of the performance data for the HVAC may be associated with the same color. As described herein, the signal 402 may be automatically segmented and clustered. In some examples, labels provided by a domain expert to the most frequent clusters may be included, where four normal operating modes corresponding to "OFF", "Heating", "Cooling with outdoor air", and "Mechanical cooling with 100% outdoor air" states may be identified.

[0094] In some examples, the example interface 400 may provide the results of such analytics. For example, the user may indicate a date range via the

"Specify Date Range" menu 426 and may begin the analytics by selecting the "Start Analytics" tab 428. In some examples, operation mode statistics 414 may be provided. One example visualization of the operation mode statistics 414 may be in the form of a pie chart 416 illustrating relative percentages of operation modes. The pie chart 416 may be associated with a legend 418 that lists the statistics. For example, the legend 418 may indicate that an "OFF" mode may be associated with 54% of the devices, a "Heating" mode may be associated with 6% of the devices, and a "Cooling with Outdoor Air" mode may be associated with 26% of the devices. Also, for example, a "Mechanical cooling with 100% outdoor air" mode may be associated with 12% of the devices. Additional and/or alternative modes may include a "Mechanical Cooling with Minimum Outdoor Air" mode, and an "Unknown ON mode". In some examples, three clusters corresponding to faulty and/or inefficient operating modes may be identified, with two clusters corresponding to

"Inefficient economization mode" and "Inefficient heating mode" categorized as medium risk, and one cluster corresponding to "Supply fan turned ON during non-working hours" categorized as high risk.

[0095] In some examples, the trained model may be deployed on a real-time system to help monitor the performance of the AHUs and detect faulty operation on a continuous basis, as shown in Figure 4. The operating modes in this system may correspond to the clusters labeled as normal behavior during the training operation, while the known faults may correspond to the clusters identified to be risky. The unknown faults may correspond to any anomalous activity identified during real-time operation. Such unknown faults may be labeled and assigned a risk category during periodic re-training.

[0096] As described herein, the example interface 400 may provide fault statistics 420, including data for "Unknown Faults" 422 and for "Known Faults" 424. In some examples, the list of "Unknown Faults" 422 may include the faults listed by date and time. In some example, the example interface 400 may provide "Known Faults" 424, including a description and a percentage occurrence. For example, a fault "Heating coil valve is fully open/supply air temp error" may have a 2.80% occurrence; a fault "Supply Air Temp should be less than return air temp" may have a 3.29% occurrence; a fault "Supply and mixed air temp should be nearly the same" may have a 1.31 % occurrence; and a fault "Persistent supply air temp error exists" may have a 2.91 % occurrence. A user may identify faulty devices and carry out remedial and/or prevention methods based on results displayed in the example interface 400.

[0097] Referring again to Figure 1 , the components of system 100 may be computing resources, each including a suitable combination of a physical computing device, a virtual computing device, a network, software, a cloud infrastructure, a hybrid cloud infrastructure that may include a first cloud infrastructure and a second cloud infrastructure that is different from the first cloud infrastructure, and so forth. The components of system 100 may be a combination of hardware and programming for performing a designated visualization function. In some instances, each component may include a processor and a memory, while programming code is stored on that memory and executable by a processor to perform a designated function. [0098] For example, the monitoring module 104 may be a combination of hardware and programming to receive a time series via a processor. Also, for example, the monitoring module 104 may include software programming to parse signals from different devices. The monitoring module 104 may include hardware to physically store, for example, such received signals, and

associations of signals with devices, locations, dates, and so forth, and/or maintain a dynamically updated database that stores the signals and such associated data.

[0099] Also, for example, the clustering module 106 may be a combination of hardware and programming to automatically segment the time series, and generate a plurality of clusters based on similar segments. Also, for example, the clustering module 106 may include software programming to identify segments, modify clusters based on the user interactions with associated cluster representatives, and associate segments to known clusters. The clustering module 106 may include hardware to physically store, for example, such clusters, and/or maintain a dynamically updated database that stores the clusters and associated data (e.g., a binary classifier) for the cluster.

[00100] Likewise, the interaction module 108 may be a combination of hardware and programming to provide cluster representatives associated with the generated clusters to a computing device via an interactive graphical user interface, and identify user interaction with at least one cluster representative. Also, for example, the interaction module 108 may include programming to generate the cluster representatives, including a rectangular display and an MDS display. In some examples, the interaction module 108 may include hardware to physically store the user interactions. Also, for example, the interaction module 108 may include software programming to dynamically interact with the other components of system 100.

[00101] Generally, the components of system 100 may include

programming and/or physical networks to be communicatively linked to other components of system 100. In some instances, the components of system 100 may include a processor and a memory, while programming code is stored and on that memory and executable by a processor to perform designated functions. [00102] A computing device, as used herein, may be, for example, a web- based server, a local area network server, a cloud-based server, a notebook computer, a desktop computer, an all-in-one system, a tablet computing device, a mobile phone, an electronic book reader, or any other electronic device suitable for provisioning a computing resource to perform a unified visualization interface. The computing device may include a processor and a computer- readable storage medium.

[0032] Figure 5 is a block diagram illustrating one example of a computer readable medium for interactive analytics on a time series. Processing system 500 includes a processor 502, a computer readable medium 508, input devices 504, and output devices 506. Processor 502, computer readable medium 508, input devices 504, and output devices 506 are coupled to each other through a communication link (e.g., a bus).

[0033] Processor 502 executes instructions included in the computer readable medium 508. Computer readable medium 508 includes time series receipt instructions 510 to receive a time series via processor 502.

[0034] Computer readable medium 508 includes segmenting and clustering instructions 512 to automatically segment the time series, and generate a plurality of clusters based on similar segments.

[0035] Computer readable medium 508 includes cluster representative providing instructions 514 to provide cluster representatives associated with the generated clusters to a computing device via an interactive graphical user interface.

[0036] Computer readable medium 508 includes user interaction identification instructions 516 to identify user interaction with at least one cluster

representative.

[0037] Computer readable medium 508 includes cluster modification instructions 518 to modify the at least one cluster representative based on the user interaction. In some examples, the cluster modification instructions 518 include instructions to modify the cluster associated with the at least one cluster representative. [0038] Computer readable medium 508 includes signal analysis model training instructions 520 to train a signal analysis model based on the user interaction.

[0039] Computer readable medium 508 includes signal analysis instructions 522 to automatically analyze another time series based on the signal analysis model.

[0040] Input devices 504 include a keyboard, mouse, data ports, and/or other suitable devices for inputting information into processing system 500. In some examples, input devices 504, such as a computing device, are used to receive user interaction with at least one cluster representative. Output devices 506 include a monitor, speakers, data ports, and/or other suitable devices for outputting information from processing system 500. In some examples, output devices 506 are used to provide the cluster representatives to a computing device via an interactive graphical user interface. In some examples, output devices 506 are used to provide an interactive fault detection interface via a computing device.

[0041] As used herein, a "computer readable medium" may be any electronic, magnetic, optical, or other physical storage apparatus to contain or store information such as executable instructions, data, and the like. For example, any computer readable storage medium described herein may be any of Random Access Memory (RAM), volatile memory, non-volatile memory, flash memory, a storage drive (e.g., a hard drive), a solid state drive, and the like, or a combination thereof. For example, the computer readable medium 508 can include one of or multiple different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; optical media such as compact disks (CDs) or digital video disks (DVDs); or other types of storage devices.

[0042] As described herein, various components of the processing system 500 are identified and refer to a combination of hardware and programming configured to perform a designated visualization function. As illustrated in Figure 2, the programming may be processor executable instructions stored on tangible computer readable medium 508, and the hardware may include processor 502 for executing those instructions. Thus, computer readable medium 508 may store program instructions that, when executed by processor 502, implement the various components of the processing system 500.

[0043] Such computer readable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture can refer to any manufactured single component or multiple components. The storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.

[0044] Computer readable medium 508 may be any of a number of memory components capable of storing instructions that can be executed by Processor 502. Computer readable medium 508 may be non-transitory in the sense that it does not encompass a transitory signal but instead is made up of one or more memory components configured to store the relevant instructions. Computer readable medium 508 may be implemented in a single device or distributed across devices. Likewise, processor 502 represents any number of processors capable of executing instructions stored by computer readable medium 508. Processor 502 may be integrated in a single device or distributed across devices. Further, computer readable medium 508 may be fully or partially integrated in the same device as processor 502 (as illustrated), or it may be separate but accessible to that device and processor 502. In some examples, computer readable medium 508 may be a machine-readable storage medium.

[0045] Figure 6 is a flow diagram illustrating one example of a method for interactive analytics on a time series. In some examples, such an example method may be implemented by a system such as, for example, system 100 of Figure 1.

[0046] At 600, a time series may be received via a processing system.

[0047] At 602, the time series may be automatically segmented, and a plurality of clusters may be generated based on similar segments. [0048] At 604, the cluster representatives associated with the generated clusters may be provided to a computing device via an interactive graphical user interface.

[0049] At 606, user interaction with at least one cluster representative may be identified.

[0050] At 608, the at least one cluster representative may be modified based on the user interaction.

[0051] At 610, labeled training data may be generated based on the user interaction.

[0052] At 612, based on the labeled training data, a signal analysis model may be trained to automatically analyze another time series.

[0053] At 614 fault detection in a monitored system may be performed based on the trained signal analysis model.

[0054] In some examples, the method may include associating the generated clusters with binary classifiers. In some examples, the method may further include receiving additional a time series, automatically segmenting the additional time series, and associating each segment with a known cluster based on an associated binary classifier.

[0055] In some examples, the method may include associating the at least one cluster representative with an alert.

[0056] In some examples, the user interaction may include at least one of merging two cluster representatives, splitting a cluster representative, and moving content from a first cluster representative to a second cluster

representative.

[0057] Examples of the disclosure provide a generalized system for interactive analytics on a time series. The generalized system enables a subject matter expert to generate binary classifiers. It may be useful for analyzing sensor data or any set of time series flows. A mean/variance mapping may be utilized to generate window features, and a Mahalanobis distance may be utilized to guide the clustering process. This may be efficient for multi-signal analysis where the nominal values generally assume a Gaussian distribution. For a more general distribution, an alternative using histograms as features, and/or KL-divergence as distances may be utilized. The generalized system may, in some examples, broadly comprise a front-end system and a back-end system. In some examples, the back-end system may be configured in Java, whereas the front- end system may be configured in HTML5 and/or JavaScript. The back-end receives a file containing signals, computes the clusters, and manages the data and analytics, whereas the front-end provides the signals and the clusters to the computing device, and enables user interaction with the analytics results (e.g., clusters).

[0058] Although specific examples have been illustrated and described herein, a variety of alternate and/or equivalent implementations may be substituted for the specific examples shown and described without departing from the scope of the present disclosure. This application is intended to cover any adaptations or variations of the specific examples discussed herein. Therefore, it is intended that this disclosure be limited only by the claims and the equivalents thereof.

Claims

1. A system comprising:

a monitoring module to receive a time series via a processor; a clustering module to automatically segment the time series, and generate a plurality of clusters based on similar segments;

an interaction module to:

provide cluster representatives associated with the generated clusters to a computing device via an interactive graphical user interface, and

identify user interaction with at least one cluster

representative; and wherein:

the clustering module is to modify the at least one cluster representative based on the user interaction; and

a machine learning module to learn from the user interaction to train a signal analysis module to automatically analyze another time series.

2. The system of claim 1 , wherein the another time series is indicative of performance data from a monitored system, and the signal analysis module is to automatically analyze the another time series for fault detection in the monitored system.

3. The system of claim 1 , further comprising a classifier module to associate the generated clusters with binary classifiers.

4. The system of claim 3, wherein the clustering module is to automatically segment the another time series, and the signal analysis module is to automatically analyze the another time series by associating each segment of the another time series with a known cluster based on an associated binary classifier.

5. The system of claim 1 , wherein the user interaction includes labeling the at least one cluster representative.

6. The system of claim 1 , wherein the user interaction includes associating the at least one cluster representative with an alert.

7. The system of claim 1 , wherein the interaction module is to provide the cluster representatives based on multidimensional scaling.

8. The system of claim 1 , wherein the clustering module is to further

distinguish between a pair of clusters based on at least one of a distance between the clusters in the pair of clusters, and a mutual information content between the pair of clusters.

9. The system of claim 1 , wherein the interactive graphical user interface is to provide at least one of a visual representation of the time series, at least one operations tab, and an interactive cluster analysis window.

10. A method comprising:

receiving a time series via a processing system;

automatically segmenting the time series, and generating a plurality of clusters based on similar segments;

providing cluster representatives associated with the generated clusters to a computing device via an interactive graphical user interface; identifying user interaction with at least one cluster representative; modifying the at least one cluster representative based on the user interaction;

generating labeled training data based on the user interaction; training, based on the labeled training data, a signal analysis model to automatically analyze another time series; and

performing fault detection in a monitored system based on the trained signal analysis model.

11.The method of claim 10, further comprising associating the generated clusters with binary classifiers.

12. The method of claim 11 , further comprising receiving additional a time series, automatically segmenting the additional time series, and associating each segment with a known cluster based on an associated binary classifier.

13. The method of claim 10, further comprising associating the at least one cluster representative with an alert.

14. The method of claim 10, wherein the user interaction includes at least one of merging two cluster representatives, splitting a cluster

representative, and moving content from a first cluster representative to a second cluster representative.

15. A non-transitory computer readable medium comprising executable

instructions to:

receive a time series via a processor;

automatically segment the time series, and generate a plurality of clusters based on similar segments;

provide cluster representatives associated with the generated clusters to a computing device via an interactive graphical user interface; identify user interaction with at least one cluster representative; modify the at least one cluster representative based on the user interaction;

train a signal analysis model based on the user interaction; and automatically analyze another time series based on the signal analysis model.