Fact-checking has become increasingly important due to the speed with which both
information and misinformation can spread in the modern media ecosystem. Therefore …

Recent work in natural language processing (NLP) has yielded appealing results from
scaling model parameters and training data; however, using only scale to improve …

As NLP models achieved state-of-the-art performances over benchmarks and gained wide
applications, it has been increasingly important to ensure the safe deployment of these …

Recent work has shown that pre-trained language models such as BERT improve
robustness to spurious correlations in the dataset. Intrigued by these results, we find that the …

Shortcut Learning of Large Language Models in Natural Language Understanding Page 1 110
COMMUNICATIONS OF THE ACM | JANUARY 2024 | VOL. 67 | NO. 1 research IMA GE B Y …

NLU models often exploit biases to achieve high dataset-specific performance without
properly learning the intended task. Recently proposed debiasing methods are shown to be …

Neural networks trained with SGD were recently shown to rely preferentially on linearly-
predictive features and can ignore complex, equally-predictive ones. This simplicity bias can …

Biased associations have been a challenge in the development of classifiers for detecting
toxic language, hindering both fairness and accuracy. As potential solutions, we investigate …

Recent studies indicate that NLU models are prone to rely on shortcut features for prediction,
without achieving true language understanding. As a result, these models fail to generalize …

Question answering (QA) models are well-known to exploit data bias, eg, the language prior
in visual QA and the position bias in reading comprehension. Recent debiasing methods …