Large models represent a groundbreaking advancement in multiple application fields,
enabling remarkable achievements across various tasks. However, their unprecedented …

This paper presents a systematic overview and comparison of parameter-efficient fine-tuning
methods covering over 40 papers published between February 2019 and February 2023 …

With the continuous growth in the number of parameters of transformer-based pretrained
language models (PLMs), particularly the emergence of large language models (LLMs) with …

Large Language Models (LLMs) possess impressive capabilities to generate meaningful
code snippets given natural language intents in zero-shot, ie, without the need for specific …

The success of large language models (LLMs), like GPT-4 and ChatGPT, has led to the
development of numerous cost-effective and accessible alternatives that are created by …

With the increasing size of pre-trained language models (PLMs), fine-tuning all the
parameters in the model is not efficient, especially when there are a large number of …

Large language models (LLMs) face the challenges in fine-tuning and deployment due to
their high memory demands and computational costs. While parameter-efficient fine-tuning …

Parameter-efficient fine-tuning (PEFT) of pre-trained language models has recently
demonstrated remarkable achievements, effectively matching the performance of full fine …

Adapting large-scale pretrained language models to downstream tasks via fine-tuning is the
standard method for achieving state-of-the-art performance on NLP benchmarks. However …

Despite the success, the process of fine-tuning large-scale PLMs brings prohibitive
adaptation costs. In fact, fine-tuning all the parameters of a colossal model and retaining …