Foundation models (FM) have demonstrated remarkable performance across a wide range
of tasks (especially in the fields of natural language processing and computer vision) …

As the size of Large Multi-Modal Models (LMMs) increases consistently, the adaptation of
these pre-trained models to specialized tasks has become a computationally and memory …

Fine-tuning is widely used as the default algorithm for transfer learning from pre-trained
models. Parameter inefficiency can however arise when, during transfer learning, all the …

Adapters are an efficient, composable alternative to full fine-tuning of pre-trained models
and help scale the deployment of large ASR models to many tasks. In practice, a task ID is …

Self-supervised speech models are a rapidly developing research topic in fake audio
detection. Many pre-trained models can serve as feature extractors, learning richer and …

In this paper, we introduce a new transfer learning approach called Adaptive Weight-space
Ensembling (AWE) that effectively adapts large pre-trained models for downstream tasks …

The breakthrough of foundation models makes foundation model fine-tuning (FMF)
workloads prevalent in modern GPU datacenters. However, existing schedulers tailored for …

Human intelligence gradually accepts new information and accumulates knowledge
throughout the lifespan. However, deep learning models suffer from a catastrophic forgetting …

Transfer learning from large multilingual pretrained models, like XLSR, has become the new
paradigm for Automatic Speech Recognition (ASR). Considering their ever-increasing size …

Adapting Automatic Speech Recognition (ASR) models to new domains leads to
Catastrophic Forgetting (CF) of previously learned information. This paper addresses CF in …