CALibrated early tIME seRies clAssifier: a method for early time series classification (ETSC).
This repository contains the reference implementation of the method described in:
J. M. Bilski, A. Jastrzębska. CALIMERA: A new early time series classification method. Information Processing and Management 60 (2023) 103465. https://www.sciencedirect.com/science/article/pii/S0306457323002029 · DOI: 10.1016/j.ipm.2023.103465
In early time series classification, a label must be assigned to an incoming time series as soon as possible, without necessarily waiting for the whole sequence. Stopping earlier usually means less accurate predictions, so a method has to balance accuracy against earliness.
CALIMERA treats this as a cost minimization problem. The cost of a prediction is:
cost = (1 - accuracy) + delay_penalty * earliness
where earliness is the average fraction of each series that was observed before the prediction was made. The delay_penalty parameter sets the trade-off: higher values make the model stop sooner, lower values make it wait for more data.
CALIMERA keeps the stopping decision separate from the classification step:
- Checkpoints. The series length is divided into (up to) 20 evenly spaced timestamps.
- Features. At each checkpoint, features are extracted from the prefix of the series seen so far using MiniRocket (very short prefixes use the raw values).
- Classifiers. A ridge classifier is trained for each checkpoint. Its leave-one-out scores are passed through a sigmoid (Platt) calibrator to estimate the probability that a prediction at that checkpoint is wrong.
- Stopping module. Going backwards from the last checkpoint, a kernel ridge regressor learns to predict the cost difference between stopping now and waiting for the next checkpoint, using the classifier scores as input.
- Inference. For each incoming series, the model stops at the first checkpoint where waiting is predicted to cost more than stopping (or at the last checkpoint) and returns that checkpoint's label.
Features of the method:
- supports binary and multiclass problems,
- supports univariate and multivariate time series,
- the stopping decision is based on calibrated estimates of classifier accuracy,
- the trade-off between accuracy and earliness is set with a single parameter.
See the article for the full description and the experimental comparison with other ETSC methods (incl. TEASER and ECONOMY).
The code was tested with Python 3.9.
pip install -r requirements.txtDependencies: sktime>=0.36, scikit-learn<=1.6, packaging.
from calimera import CALIMERA
# X_train, X_test: numpy arrays of shape (n_samples, n_channels, series_length)
# y_train: array of labels of shape (n_samples,)
model = CALIMERA(delay_penalty=1.0)
model.fit(X_train, y_train)
# for each test sample: the timestamp at which the model stopped and the predicted label
stop_timestamps, y_pred = model.test(X_test)All series must have the same length. A univariate dataset should be passed with n_channels = 1.
usage_example.py trains and evaluates CALIMERA on the multivariate Cricket dataset from the UEA & UCR Time Series Classification Repository.
-
Download the multivariate archive (
.tsformat) from timeseriesclassification.com and extract it so that the files are atMultivariate_ts/Cricket/Cricket_TRAIN.tsandMultivariate_ts/Cricket/Cricket_TEST.ts. -
Run:
python usage_example.py
The script prints the accuracy, earliness and cost on the test set.
If you use CALIMERA in your research, please cite:
@article{bilski2023calimera,
title = {CALIMERA: A new early time series classification method},
author = {Bilski, Jakub Michal and Jastrz{\k{e}}bska, Agnieszka},
journal = {Information Processing \& Management},
volume = {60},
pages = {103465},
year = {2023},
doi = {10.1016/j.ipm.2023.103465}
}