Recent works have shown how the reasoning capabilities of Large Language Models
(LLMs) can be applied to domains beyond natural language processing, such as planning …

Language understanding research is held back by a failure to relate language to the
physical world it describes and to the social interactions it facilitates. Despite the incredible …

A long-term goal of AI research is to build intelligent agents that can communicate with
humans in natural language, perceive the environment, and perform real-world tasks. Vision …

Recent advances in the areas of multimodal machine learning and artificial intelligence (AI)
have led to the development of challenging tasks at the intersection of Computer Vision …

We study the problem of learning a range of vision-based manipulation tasks from a large
offline dataset of robot interaction. In order to accomplish this, humans need easy and …

When humans design cost or goal specifications for robots, they often produce specifications
that are ambiguous, underspecified, or beyond planners' ability to solve. In these cases …

Natural language provides an accessible and expressive interface to specify long-term tasks
for robotic agents. However, non-experts are likely to specify such tasks with high-level …

In recent years, much progress has been made in learning robotic manipulation policies that
follow natural language instructions. Such methods typically learn from corpora of robot …

Abstract Recently emerged Vision-and-Language Navigation (VLN) tasks have drawn
significant attention in both computer vision and natural language processing communities …

We study the challenging problem of releasing a robot in a previously unseen environment,
and having it follow unconstrained natural language navigation instructions. Recent work on …