A comprehensive survey of continual learning: theory, method and application

L Wang, X Zhang, H Su, J Zhu - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
To cope with real-world dynamics, an intelligent system needs to incrementally acquire,
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …

A comprehensive survey of forgetting in deep learning beyond continual learning

Z Wang, E Yang, L Shen… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Forgetting refers to the loss or deterioration of previously acquired knowledge. While
existing surveys on forgetting have primarily focused on continual learning, forgetting is a …

Co2l: Contrastive continual learning

H Cha, J Lee, J Shin - Proceedings of the IEEE/CVF …, 2021 - openaccess.thecvf.com
Recent breakthroughs in self-supervised learning show that such algorithms learn visual
representations that can be transferred better to unseen tasks than cross-entropy based …

Online continual learning in image classification: An empirical survey

Z Mai, R Li, J Jeong, D Quispe, H Kim, S Sanner - Neurocomputing, 2022 - Elsevier
Online continual learning for image classification studies the problem of learning to classify
images from an online stream of data and tasks, where tasks may include new classes …

Remember the past: Distilling datasets into addressable memories for neural networks

Z Deng, O Russakovsky - Advances in Neural Information …, 2022 - proceedings.neurips.cc
We propose an algorithm that compresses the critical information of a large dataset into
compact addressable memories. These memories can then be recalled to quickly re-train a …

Forget-free continual learning with winning subnetworks

H Kang, RJL Mina, SRH Madjid… - International …, 2022 - proceedings.mlr.press
Abstract Inspired by Lottery Ticket Hypothesis that competitive subnetworks exist within a
dense network, we propose a continual learning method referred to as Winning …

Meta-learning approaches for learning-to-learn in deep learning: A survey

Y Tian, X Zhao, W Huang - Neurocomputing, 2022 - Elsevier
Compared to traditional machine learning, deep learning can learn deeper abstract data
representation and understand scattered data properties. It has gained considerable …

An empirical investigation of the role of pre-training in lifelong learning

SV Mehta, D Patil, S Chandar, E Strubell - Journal of Machine Learning …, 2023 - jmlr.org
The lifelong learning paradigm in machine learning is an attractive alternative to the more
prominent isolated learning scheme not only due to its resemblance to biological learning …

Continual learning of natural language processing tasks: A survey

Z Ke, B Liu - arXiv preprint arXiv:2211.12701, 2022 - arxiv.org
Continual learning (CL) is a learning paradigm that emulates the human capability of
learning and accumulating knowledge continually without forgetting the previously learned …

Advances and challenges in meta-learning: A technical review

A Vettoruzzo, MR Bouguelia… - … on Pattern Analysis …, 2024 - ieeexplore.ieee.org
Meta-learning empowers learning systems with the ability to acquire knowledge from
multiple tasks, enabling faster adaptation and generalization to new tasks. This review …