8000 GitHub - JakubBilski/CALIMERA: An Early Time Series Classification method · GitHub
[go: up one dir, main page]

Skip to content
10BC8
JakubBilskiPublic

About

An Early Time Series Classification method

Resources

Stars

8 stars

Watchers

2 watching

Forks

Latest commit

 

History

6 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

CALIMERA

CALibrated early tIME seRies clAssifier: a method for early time series classification (ETSC).

This repository contains the reference implementation of the method described in:

J. M. Bilski, A. Jastrzębska. CALIMERA: A new early time series classification method. Information Processing and Management 60 (2023) 103465. https://www.sciencedirect.com/science/article/pii/S0306457323002029 · DOI: 10.1016/j.ipm.2023.103465

What is early time series classification?

In early time series classification, a label must be assigned to an incoming time series as soon as possible, without necessarily waiting for the whole sequence. Stopping earlier usually means less accurate predictions, so a method has to balance accuracy against earliness.

CALIMERA treats this as a cost minimization problem. The cost of a prediction is:

cost = (1 - accuracy) + delay_penalty * earliness

where earliness is the average fraction of each series that was observed before the prediction was made. The delay_penalty parameter sets the trade-off: higher values make the model stop sooner, lower values make it wait for more data.

How it works

CALIMERA keeps the stopping decision separate from the classification step:

  1. Checkpoints. The series length is divided into (up to) 20 evenly spaced timestamps.
  2. Features. At each checkpoint, features are extracted from the prefix of the series seen so far using MiniRocket (very short prefixes use the raw values).
  3. Classifiers. A ridge classifier is trained for each checkpoint. Its leave-one-out scores are passed through a sigmoid (Platt) calibrator to estimate the probability that a prediction at that checkpoint is wrong.
  4. Stopping module. Going backwards from the last checkpoint, a kernel ridge regressor learns to predict the cost difference between stopping now and waiting for the next checkpoint, using the classifier scores as input.
  5. Inference. For each incoming series, the model stops at the first checkpoint where waiting is predicted to cost more than stopping (or at the last checkpoint) and returns that checkpoint's label.

Features of the method:

  • supports binary and multiclass problems,
  • supports univariate and multivariate time series,
  • the stopping decision is based on calibrated estimates of classifier accuracy,
  • the trade-off between accuracy and earliness is set with a single parameter.

See the article for the full description and the experimental comparison with other ETSC methods (incl. TEASER and ECONOMY).

Installation

The code was tested with Python 3.9.

pip install -r requirements.txt

Dependencies: sktime>=0.36, scikit-learn<=1.6, packaging.

Usage

from calimera import CALIMERA

# X_train, X_test: numpy arrays of shape (n_samples, n_channels, series_length)
# y_train: array of labels of shape (n_samples,)
model = CALIMERA(delay_penalty=1.0)
model.fit(X_train, y_train)

# for each test sample: the timestamp at which the model stopped and the predicted label
stop_timestamps, y_pred = model.test(X_test)

All series must have the same length. A univariate dataset should be passed with n_channels = 1.

Running the example

usage_example.py trains and evaluates CALIMERA on the multivariate Cricket dataset from the UEA & UCR Time Series Classification Repository.

  1. Download the multivariate archive (.ts format) from timeseriesclassification.com and extract it so that the files are at Multivariate_ts/Cricket/Cricket_TRAIN.ts and Multivariate_ts/Cricket/Cricket_TEST.ts.

  2. Run:

    python usage_example.py

The script prints the accuracy, earliness and cost on the test set.

Citation

If you use CALIMERA in your research, please cite:

@article{bilski2023calimera,
  title   = {CALIMERA: A new early time series classification method},
  author  = {Bilski, Jakub Michal and Jastrz{\k{e}}bska, Agnieszka},
  journal = {Information Processing \& Management},
  volume  = {60},
  pages   = {103465},
  year    = {2023},
  doi     = {10.1016/j.ipm.2023.103465}
}

About

An Early Time Series Classification method

Resources

Stars

8 stars

Watchers

2 watching

Forks

Releases

Packages

Used by

Contributors

Languages

0