When transferring a pretrained model to a downstream task, two popular methods are full
fine-tuning (updating all the model parameters) and linear probing (updating only the last …

Recently, a boom of papers has shown extraordinary progress in zero-shot and few-shot
learning with various prompt-based models. It is commonly argued that prompts help models …

Shortcut Learning of Large Language Models in Natural Language Understanding Page 1 110
COMMUNICATIONS OF THE ACM | JANUARY 2024 | VOL. 67 | NO. 1 research IMA GE B Y …

Natural Language Inference (NLI) has been extensively studied by the NLP community as a
framework for estimating the semantic relation between sentence pairs. While early work …

Robust fine-tuning aims to achieve competitive in-distribution (ID) performance while
maintaining the out-of-distribution (OOD) robustness of a pre-trained model when …

Abstract Machine learning (ML) systems in natural language processing (NLP) face
significant challenges in generalizing to out-of-distribution (OOD) data, where the test …

We advocate the importance of exposing uncertainty on results of language model
prompting which display bias modes resembling cognitive biases, and propose to help …

Prompt-based fine-tuning has boosted the performance of Pre-trained Language Models
(PLMs) on few-shot text classification by employing task-specific prompts. Yet, PLMs are …

Natural language understanding (NLU) models often rely on dataset biases rather than
intended task-relevant features to achieve high performance on specific datasets. As a …

Discrete prompts have been used for fine-tuning Pre-trained Language Models for diverse
NLP tasks. In particular, automatic methods that generate discrete prompts from a small set …