Foundational challenges in assuring alignment and safety of large language models
This work identifies 18 foundational challenges in assuring the alignment and safety of large
language models (LLMs). These challenges are organized into three different categories …
language models (LLMs). These challenges are organized into three different categories …
Gated linear attention transformers with hardware-efficient training
Transformers with linear attention allow for efficient parallel training but can simultaneously
be formulated as an RNN with 2D (matrix-valued) hidden states, thus enjoying linear (with …
be formulated as an RNN with 2D (matrix-valued) hidden states, thus enjoying linear (with …
Benchmarks as microscopes: A call for model metrology
Modern language models (LMs) pose a new challenge in capability assessment. Static
benchmarks inevitably saturate without providing confidence in the deployment tolerances …
benchmarks inevitably saturate without providing confidence in the deployment tolerances …
LLM-rubric: A multidimensional, calibrated approach to automated evaluation of natural language texts
This paper introduces a framework for the automated evaluation of natural language texts. A
manually constructed rubric describes how to assess multiple dimensions of interest. To …
manually constructed rubric describes how to assess multiple dimensions of interest. To …
What We Talk About When We Talk About LMs: Implicit Paradigm Shifts and the Ship of Language Models
S Zhu, JM Rzeszotarski - arXiv preprint arXiv:2407.01929, 2024 - arxiv.org
The term Language Models (LMs), as a time-specific collection of models of interest, is
constantly reinvented, with its referents updated much like the $\textit {Ship of Theseus} …
constantly reinvented, with its referents updated much like the $\textit {Ship of Theseus} …
Evaluation and Continual Improvement for an Enterprise AI Assistant
The development of conversational AI assistants is an iterative process with multiple
components. As such, the evaluation and continual improvement of these assistants is a …
components. As such, the evaluation and continual improvement of these assistants is a …
Natural Language Processing RELIES on Linguistics
Large Language Models (LLMs) have become capable of generating highly fluent text in
certain languages, without modules specially designed to capture grammar or semantic …
certain languages, without modules specially designed to capture grammar or semantic …
Balancing specialization and adaptation in a transforming scientific landscape
L Gautheron - EPJ Data Science, 2025 - epjds.epj.org
How do scientists navigate between the need to capitalize on their prior knowledge through
specialization, and the urge to adapt to evolving research opportunities? Drawing from …
specialization, and the urge to adapt to evolving research opportunities? Drawing from …
Towards Compositionally Generalizable Semantic Parsing in Large Language Models: A Survey
A Mannekote - arXiv preprint arXiv:2404.13074, 2024 - arxiv.org
Compositional generalization is the ability of a model to generalize to complex, previously
unseen types of combinations of entities from just having seen the primitives. This type of …
unseen types of combinations of entities from just having seen the primitives. This type of …
Brains Over Brawn: Small AI Labs in the Age of Datacenter-Scale Compute
The prevailing trend towards large models that demand extensive computational resources
threatens to marginalize smaller research labs, constraining innovation and diversity in the …
threatens to marginalize smaller research labs, constraining innovation and diversity in the …