With the continuous growth in the number of parameters of transformer-based pretrained
language models (PLMs), particularly the emergence of large language models (LLMs) with …

Parameter-efficient fine-tuning (PEFT) methods seek to adapt large models via updates to a
small number of weights. However, much prior interpretability work has shown that …

Large foundation models are becoming ubiquitous, but training them from scratch is
prohibitively expensive. Thus, efficiently adapting these powerful models to downstream …

As the parameters of LLMs expand, the computational cost of fine-tuning the entire model
becomes prohibitive. To address this challenge, we introduce a PEFT method, Principal …

Fine-tuning techniques based on Large Pretrained Language Models (LPLMs) have been
proven to significantly enhance model performance on a variety of downstream tasks and …

Low-rank adaptation (LoRA) is a natural method for finetuning in communication-
constrained machine learning settings such as cross-device federated learning. Prior work …

Current parameter-efficient fine-tuning (PEFT) methods build adapters without considering
the context of downstream task to learn, or the context of important knowledge to maintain …

Artificial Intelligence (AI) Foundation Models (FMs), pre-trained on massive datasets, have
recently emerged as a pivotal asset in a wide array of tasks. Examples of FMs include Large …

In response to the challenges posed by the extensive parameter updates required for full
fine-tuning of large-scale pre-trained models, parameter-efficient fine-tuning (PEFT) …

Fine-tuning large-scale pre-trained models via transfer learning is an emerging important
paradigm for a wide range of downstream tasks, with performance heavily reliant on …