Existing fine-tuning methods either tune all parameters of the pre-trained model (full fine-
tuning), which is not efficient, or only tune the last linear layer (linear probing), which suffers …

With the increasing size of pre-trained language models (PLMs), fine-tuning all the
parameters in the model is not efficient, especially when there are a large number of …

As a dominant paradigm, fine-tuning a pre-trained model on the target data is widely used in
many deep learning applications, especially for small data sets. However, recent studies …

Fine-tuning from pre-trained ImageNet models has become the de-facto standard for various
computer vision tasks. Current practices for fine-tuning typically involve selecting an ad-hoc …

Among the widely used parameter-efficient finetuning (PEFT) methods, LoRA and its
variants have gained considerable popularity because of avoiding additional inference …

Fine-tuning pre-trained deep networks on a small dataset is an important component in the
deep learning pipeline. A critical problem in fine-tuning is how to avoid over-fitting when …

Abstract We proposed a Prior Gradient Mask Guided Pruning-aware Fine-Tuning (PGMPF)
framework to accelerate deep Convolutional Neural Networks (CNNs). In detail, the …

Abstract Visual Parameter-Efficient Fine-Tuning (PEFT) has become a powerful alternative
for full fine-tuning so as to adapt pre-trained vision models to downstream tasks, which only …

Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks
requires updating hundreds of millions to billions of parameters, and storing a large copy of …

Fine-tuning pre-trained models has been ubiquitously proven to be effective in a wide range
of NLP tasks. However, fine-tuning the whole model is parameter inefficient as it always …