Abstract Evaluation practices in natural language generation (NLG) have many known flaws,
but improved evaluation approaches are rarely widely adopted. This issue has become …

In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large
language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine …

Finetuning language models on a collection of datasets phrased as instructions has been
shown to improve model performance and generalization to unseen tasks. In this paper we …

Generative AI systems across modalities, ranging from text, image, audio, and video, have
broad social impacts, but there exists no official standard for means of evaluating those …

Understanding the landscape of potential harms from algorithmic systems enables
practitioners to better anticipate consequences of the systems they build. It also supports the …

As the reach of large language models (LMs) expands globally, their ability to cater to
diverse cultural contexts becomes crucial. Despite advancements in multilingual …

Deep neural networks often learn unintended bias during training, which might have harmful
effects when deployed in realworld settings. This work surveys 214 papers related to …

With language models becoming increasingly ubiquitous, it has become essential to
address their inequitable treatment of diverse demographic groups and factors. Most …

Bias research in NLP seeks to analyse models for social biases, thus helping NLP
practitioners uncover, measure, and mitigate social harms. We analyse the body of work that …

With rapid development and deployment of generative language models in global settings,
there is an urgent need to also scale our measurements of harm, not just in the number and …