Learning from unstructured and uncurated data has become the dominant paradigm for
generative approaches in language or vision. Such unstructured and unguided behavior …

Recent work has described neural-network-based agents that are trained with reinforcement
learning (RL) to execute language-like commands in simulated worlds, as a step towards an …

It is highly desirable for robots that work alongside humans to be able to understand
instructions in natural language. Existing language conditioned imitation learning models …

Traditional approaches in physics-based motion generation centered around imitation
learning and reward shaping often struggle to adapt to new scenarios. To tackle this …

We study how visual representations pre-trained on diverse human video data can enable
data-efficient learning of downstream robotic manipulation tasks. Concretely, we pre-train a …

Humans interpret scenes by recognizing both the identities and positions of objects in their
observations. For a robot to perform tasks such as\enquote {pick and place}, understanding …

General-purpose robotic systems must master a large repertoire of diverse skills. While
reinforcement learning provides a powerful framework for acquiring individual behaviors, the …

Prompt-based learning has emerged as a successful paradigm in natural language
processing, where a single general-purpose language model can be instructed to perform …

Large transformer-based architectures are capable of complex robot task planning and low-
level control. In the natural language processing (NLP) community, fine-tuning large …

Generalizable articulated object manipulation is essential for home-assistant robots. Recent
efforts focus on imitation learning from demonstrations or reinforcement learning in …