High-level human instructions often correspond to behaviors with multiple implicit steps. In
order for robots to be useful in the real world, they must be able to to reason over both …

Learning a generalist embodied agent capable of completing multiple tasks poses
challenges, primarily stemming from the scarcity of action-labeled robotic datasets. In …

Modern model-free reinforcement learning methods have recently demonstrated impressive
results on a number of problems. However, complex domains like dexterous manipulation …

Generating human-like behavior on robots is a great challenge especially in dexterous
manipulation tasks with robotic hands. Scripting policies from scratch is intractable due to …

Learning predictive models from interaction with the world allows an agent, such as a robot,
to learn about how the world works, and then use this learned model to plan coordinated …

Abstract We present Language-Image Value learning (LIV), a unified objective for vision-
language representation and reward learning from action-free videos with text annotations …

Prompt-based learning has been demonstrated as a compelling paradigm contributing to
large language models' tremendous success (LLMs). Inspired by their success in language …

The manner in which humans learn, plan, and decide actions is a very compelling subject.
Moreover, the mechanism behind high-level cognitive functions, such as action planning …

A rich representation is key to general robotic manipulation, but existing approaches to
representation learning require large amounts of multimodal demonstrations. In this work we …

While natural language offers a convenient shared interface for humans and robots,
enabling robots to interpret and follow language commands remains a longstanding …