Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated textual
explanations. These explanations are used downstream in three ways: as data …

Natural language generation has witnessed significant advancements due to the training of
large language models on vast internet-scale datasets. Despite these advancements, there …

Human variation in labeling is often considered noise. Annotation projects for machine
learning (ML) aim at minimizing human label variation, with the assumption to maximize …

Abstract Many tasks in Natural Language Processing (NLP) and Computer Vision (CV) offer
evidence that humans disagree, from objective tasks such as part-of-speech tagging to more …

Ambiguity is an intrinsic feature of natural language. Managing ambiguity is a key part of
human language understanding, allowing us to anticipate misunderstanding as …

We investigate how disagreement in natural language inference (NLI) annotation arises. We
developed a taxonomy of disagreement sources with 10 categories spanning 3 high-level …

Humans produce and consume language in a particular cultural context, which includes
knowledge about specific norms and practices. A listener's awareness of the cultural context …

To quantify how well natural language understanding models can capture consistency in a
general conversation, we introduce the DialoguE COntradiction DEtection task (DECODE) …

In Large Language Models (LLMs), there have been consistent advancements in task-
specific performance, largely influenced by effective prompt design. While recent research …

Calibration is a popular framework to evaluate whether a classifier knows when it does not
know-ie, its predictive probabilities are a good indication of how likely a prediction is to be …