Automated software engineering has been greatly empowered by the recent advances in
Large Language Models (LLMs) for programming. While current benchmarks have shown …

Language models are aligned to emulate the collective voice of many, resulting in outputs
that align with no one in particular. Steering LLMs away from generic output is possible …

Music has a unique and complex structure which is challenging for both expert humans and
existing AI systems to understand, and presents unique challenges relative to other forms of …

The ability to build and leverage world models is essential for a general-purpose AI agent.
Testing such capabilities is hard, in part because the building blocks of world models are ill …

Keyword mnemonics are memorable explanations that link new terms to simpler keywords.
Prior works generate mnemonics for students, but they do not guide models toward …

When answering questions, LLMs can convey not only an answer, but a level of confidence
about the answer being correct. This includes explicit confidence markers (eg giving a …

Experts in various fields routinely perform methodical writing tasks to plan, organize, and
report their work. From a clinician writing a differential diagnosis for a patient, to a teacher …

Although Large Language Models (LLMs) are becoming increasingly powerful, they still
exhibit significant but subtle weaknesses, such as mistakes in instruction-following or coding …

Large language models (LLMs) have shown promising abilities as cost-effective and
reference-free evaluators for assessing language generation quality. In particular, pairwise …

Human evaluation of generated language through pairwise preference judgments is
pervasive. However, under common scenarios, such as when generations from a model pair …