This article surveys the use of natural language in robotics from a robotics point of view. To
use human language, robots must map words to aspects of the physical world, mediated by …

Machine learning has long since become a keystone technology, accelerating science and
applications in a broad range of domains. Consequently, the notion of applying learning …

Large language models (LLMs) trained on code-completion have been shown to be capable
of synthesizing simple Python programs from docstrings [1]. We find that these code-writing …

Language models demonstrate both quantitative improvement and new qualitative
capabilities with increasing scale. Despite their potentially transformative impact, these new …

Transformers have revolutionized vision and natural language processing with their ability to
scale with large datasets. But in robotic manipulation, data is both limited and expensive …

Large language models (LLMs) are shown to possess a wealth of actionable knowledge that
can be extracted for robot manipulation in the form of reasoning and planning. Despite the …

How can we imbue robots with the ability to manipulate objects precisely but also to reason
about them in terms of abstract concepts? Recent works in manipulation have shown that …

Abstract We present ALFRED (Action Learning From Realistic Environments and Directives),
a benchmark for learning a mapping from natural language instructions and egocentric …

Systems for language-guided human-robot interaction must satisfy two key desiderata for
broad adoption: adaptivity and learning efficiency. Unfortunately, existing instruction …

Natural language provides an accessible and expressive interface to specify long-term tasks
for robotic agents. However, non-experts are likely to specify such tasks with high-level …