Generative AI systems across modalities, ranging from text, image, audio, and video, have
broad social impacts, but there exists no official standard for means of evaluating those …

Alongside an explosion in research and development related to large language models,
there has been a concomitant rise in the creation of pretraining datasets—massive …

AI progress is creating a growing range of risks and opportunities, but it is often unclear how
they should be navigated. In many cases, the barriers and uncertainties faced are at least …

Machine learning (ML) datasets, often perceived as neutral, inherently encapsulate abstract
and disputed social constructs. Dataset curators frequently employ value-laden terms such …

Current Large Language Models (LLMs) are unparalleled in their ability to generate
grammatically correct, fluent text. LLMs are appearing rapidly, and debates on LLM …

Recent work has advocated for training AI models on ever-larger datasets, arguing that as
the size of a dataset increases, the performance of a model trained on that dataset will …

AI models are increasingly prevalent in high-stakes environments, necessitating thorough
assessment of their capabilities and risks. Benchmarks are popular for measuring these …

This review gives an extensive overview of evaluation methods for task-oriented dialogue
systems, paying special attention to practical applications of dialogue systems, for example …

The integrity of AI benchmarks is fundamental to accurately assess the capabilities of AI
systems. The internal validity of these benchmarks-ie, making sure they are free from …

Entity-and event-level conceptualization, as fundamental elements of human cognition,
plays a pivotal role in generalizable reasoning. This process involves abstracting specific …