Pytorch code for the paper: EEC: Learning to Encode and Regenerate Images for Continual Learning

The two main impediments to continual learning are catastrophic forgetting and memory limitations on the storage of data. To cope with these challenges, we propose a novel, cognitively-inspired approach which trains autoencoders with Neural Style Transfer to encode and store images. During training on a new task, reconstructed images from encoded episodes are replayed in order to avoid catastrophic forgetting. The loss function for the reconstructed images is weighted to reduce its effect during classifier training to cope with image degradation. When the system runs out of memory the encoded episodes are converted into centroids and covariance matrices, which are used to generate pseudo-images during classifier training, keeping classifier performance stable while using less memory. Our approach increases classification accuracy by 13-17% over state-of-the-art methods on benchmark datasets, while requiring 78% less storage space.

• torch (Currently working with 1.3.1)
• Scipy (Currently working with 1.2.1)
• Scikit Learn (Currently working with 0.21.2)
• Use requirements.txt to install all the required libraries
• Download the datasets in */data directory

• Create checkpoint, data and previous_classes folders.
• Run multiple_auto_decay.py to run EEC with multiple autoencoders without using pseudorehearsal.
• Run multiple_pseudo.py to run EEC with multiple autoencoders with pseudorehearsal.
• The code currently has parameters set for ImageNet-50. Just change the appropriate parameters to run it on other datasets.
• Label smoothing was used from this repo: Link

@inproceedings{
ayub2021eec,
title={{EEC}: Learning to Encode and Regenerate Images for Continual Learning},
author={Ali Ayub and Alan R. Wagner},
booktitle={International Conference on Learning Representations},
year={2021}
}