Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need
Class-incremental learning (CIL) aims to adapt to emerging new classes without forgetting
old ones. Traditional CIL models are trained from scratch to continually acquire knowledge …
old ones. Traditional CIL models are trained from scratch to continually acquire knowledge …
Class-incremental learning: A survey
Deep models, eg, CNNs and Vision Transformers, have achieved impressive achievements
in many vision tasks in the closed world. However, novel classes emerge from time to time in …
in many vision tasks in the closed world. However, novel classes emerge from time to time in …
Ranpac: Random projections and pre-trained models for continual learning
MD McDonnell, D Gong, A Parvaneh… - Advances in …, 2024 - proceedings.neurips.cc
Continual learning (CL) aims to incrementally learn different tasks (such as classification) in
a non-stationary data stream without forgetting old ones. Most CL works focus on tackling …
a non-stationary data stream without forgetting old ones. Most CL works focus on tackling …
Speciality vs generality: An empirical study on catastrophic forgetting in fine-tuning foundation models
Foundation models, including Vision Language Models (VLMs) and Large Language
Models (LLMs), possess the $ generality $ to handle diverse distributions and tasks, which …
Models (LLMs), possess the $ generality $ to handle diverse distributions and tasks, which …
CLR: Channel-wise lightweight reprogramming for continual learning
Continual learning aims to emulate the human ability to continually accumulate knowledge
over sequential tasks. The main challenge is to maintain performance on previously learned …
over sequential tasks. The main challenge is to maintain performance on previously learned …
Balancing speciality and versatility: a coarse to fine framework for supervised fine-tuning large language model
Aligned Large Language Models (LLMs) showcase remarkable versatility, capable of
handling diverse real-world tasks. Meanwhile, aligned LLMs are also expected to exhibit …
handling diverse real-world tasks. Meanwhile, aligned LLMs are also expected to exhibit …
Investigating forgetting in pre-trained representations through continual learning
Representation forgetting refers to the drift of contextualized representations during
continual training. Intuitively, the representation forgetting can influence the general …
continual training. Intuitively, the representation forgetting can influence the general …
On the usage of continual learning for out-of-distribution generalization in pre-trained language models of code
Pre-trained language models (PLMs) have become a prevalent technique in deep learning
for code, utilizing a two-stage pre-training and fine-tuning procedure to acquire general …
for code, utilizing a two-stage pre-training and fine-tuning procedure to acquire general …
Premonition: Using generative models to preempt future data changes in continual learning
Continual learning requires a model to adapt to ongoing changes in the data distribution,
and often to the set of tasks to be performed. It is rare, however, that the data and task …
and often to the set of tasks to be performed. It is rare, however, that the data and task …
Information Bottleneck Based Data Correction in Continual Learning
Continual Learning (CL) requires model to retain previously learned knowledge while
learning new tasks. Recently, experience replay-based methods have made significant …
learning new tasks. Recently, experience replay-based methods have made significant …