A comprehensive survey of continual learning: Theory, method and application

L Wang, X Zhang, H Su, J Zhu - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
To cope with real-world dynamics, an intelligent system needs to incrementally acquire,
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …

[HTML][HTML] Embracing change: Continual learning in deep neural networks

R Hadsell, D Rao, AA Rusu, R Pascanu - Trends in cognitive sciences, 2020 - cell.com
Artificial intelligence research has seen enormous progress over the past few decades, but it
predominantly relies on fixed datasets and stationary environments. Continual learning is an …

Three types of incremental learning

GM Van de Ven, T Tuytelaars, AS Tolias - Nature Machine Intelligence, 2022 - nature.com
Incrementally learning new information from a non-stationary stream of data, referred to as
'continual learning', is a key feature of natural intelligence, but a challenging problem for …

Dataset distillation using neural feature regression

Y Zhou, E Nezhadarya, J Ba - Advances in Neural …, 2022 - proceedings.neurips.cc
Dataset distillation aims to learn a small synthetic dataset that preserves most of the
information from the original dataset. Dataset distillation can be formulated as a bi-level …

Class-incremental learning: survey and performance evaluation on image classification

M Masana, X Liu, B Twardowski… - … on Pattern Analysis …, 2022 - ieeexplore.ieee.org
For future learning systems, incremental learning is desirable because it allows for: efficient
resource usage by eliminating the need to retrain from scratch at the arrival of new data; …

Online continual learning through mutual information maximization

Y Guo, B Liu, D Zhao - International Conference on Machine …, 2022 - proceedings.mlr.press
This paper proposed a new online continual learning approach called OCM based on
mutual information (MI) maximization. It achieves two objectives that are critical in dealing …

Online continual learning in image classification: An empirical survey

Z Mai, R Li, J Jeong, D Quispe, H Kim, S Sanner - Neurocomputing, 2022 - Elsevier
Online continual learning for image classification studies the problem of learning to classify
images from an online stream of data and tasks, where tasks may include new classes …

Gdumb: A simple approach that questions our progress in continual learning

A Prabhu, PHS Torr, PK Dokania - … , Glasgow, UK, August 23–28, 2020 …, 2020 - Springer
We discuss a general formulation for the Continual Learning (CL) problem for classification—
a learning task where a stream provides samples to a learner and the goal of the learner …

Brain-inspired replay for continual learning with artificial neural networks

GM Van de Ven, HT Siegelmann, AS Tolias - Nature communications, 2020 - nature.com
Artificial neural networks suffer from catastrophic forgetting. Unlike humans, when these
networks are trained on something new, they rapidly forget what was learned before. In the …

Computationally budgeted continual learning: What does matter?

A Prabhu, HA Al Kader Hammoud… - Proceedings of the …, 2023 - openaccess.thecvf.com
Continual Learning (CL) aims to sequentially train models on streams of incoming data that
vary in distribution by preserving previous knowledge while adapting to new data. Current …