A comprehensive survey of forgetting in deep learning beyond continual learning

Z Wang, E Yang, L Shen, H Huang - arXiv preprint arXiv:2307.09218, 2023 - arxiv.org
Forgetting refers to the loss or deterioration of previously acquired information or knowledge.
While the existing surveys on forgetting have primarily focused on continual learning …

Boosting continual learning of vision-language models via mixture-of-experts adapters

J Yu, Y Zhuge, L Zhang, P Hu… - Proceedings of the …, 2024 - openaccess.thecvf.com
Continual learning can empower vision-language models to continuously acquire new
knowledge without the need for access to the entire historical dataset. However mitigating …

Expandable subspace ensemble for pre-trained model-based class-incremental learning

DW Zhou, HL Sun, HJ Ye… - Proceedings of the IEEE …, 2024 - openaccess.thecvf.com
Abstract Class-Incremental Learning (CIL) requires a learning system to continually learn
new classes without forgetting. Despite the strong performance of Pre-Trained Models …

A survey on knowledge distillation of large language models

X Xu, M Li, C Tao, T Shen, R Cheng, J Li, C Xu… - arXiv preprint arXiv …, 2024 - arxiv.org
This survey presents an in-depth exploration of knowledge distillation (KD) techniques
within the realm of Large Language Models (LLMs), spotlighting the pivotal role of KD in …

Catastrophic forgetting in deep learning: A comprehensive taxonomy

EL Aleixo, JG Colonna, M Cristo… - arXiv preprint arXiv …, 2023 - arxiv.org
Deep Learning models have achieved remarkable performance in tasks such as image
classification or generation, often surpassing human accuracy. However, they can struggle …

Incremental Nuclei Segmentation from Histopathological Images via Future-class Awareness and Compatibility-inspired Distillation

H Wang, H Wu, J Qin - … of the IEEE/CVF Conference on …, 2024 - openaccess.thecvf.com
We present a novel semantic segmentation approach for incremental nuclei segmentation
from histopathological images which is a very challenging task as we have to incrementally …

Hyper-feature aggregation and relaxed distillation for class incremental learning

R Wu, H Liu, Z Yue, JB Li, CW Sham - Pattern Recognition, 2024 - Elsevier
Although neural networks have been used extensively in pattern recognition scenarios, the
pre-acquisition of datasets is still challenging. In most pattern recognition areas, preparing a …

FCS: Feature Calibration and Separation for Non-Exemplar Class Incremental Learning

Q Li, Y Peng, J Zhou - … of the IEEE/CVF Conference on …, 2024 - openaccess.thecvf.com
Abstract Non-Exemplar Class Incremental Learning (NECIL) involves learning a
classification model on a sequence of data without access to exemplars from previously …

Multi-scale feature decoupling and similarity distillation for class-incremental defect detection of photovoltaic cells

S Wang, H Chen, Z Zhang, B Su - Measurement, 2024 - Elsevier
Existing vision-based photovoltaic cell defect detection methods usually update models with
all defect data of both old and new categories to adapt to new classes emerging in the …

MoBoo: Memory-Boosted Vision Transformer for Class-Incremental Learning

B Ni, X Nie, C Zhang, S Xu, X Zhang… - … on Circuits and …, 2024 - ieeexplore.ieee.org
Continual learning strives to acquire knowledge across sequential tasks without forgetting
previously assimilated knowledge. Current state-of-the-art methodologies utilize dynamic …