Sequential decision making, commonly formalized as Markov Decision Process (MDP)
optimization, is an important challenge in artificial intelligence. Two key approaches to this …

We propose to forecast future hand-object interactions given an egocentric video. Instead of
predicting action labels or pixels, we directly predict the hand motion trajectory and the …

Everyday tasks of long-horizon and comprising a sequence of multiple implicit subtasks still
impose a major challenge in offline robot control. While a number of prior methods aimed to …

Recognizing and generating object-state compositions has been a challenging task,
especially when generalizing to unseen compositions. In this paper, we study the task of …

Future prediction is a fundamental principle of intelligence that helps plan actions and avoid
possible dangers. As the future is uncertain to a large extent, modeling the uncertainty and …

Video prediction models combined with planning algorithms have shown promise in
enabling robots to learn to perform many vision-based tasks through only self-supervision …

Robotic reinforcement learning (RL) holds the promise of enabling robots to learn complex
behaviors through experience. However, realizing this promise for long-horizon tasks in the …

Many animals, and an increasing number of artificial agents, display sophisticated
capabilities to perceive and manipulate objects. But human beings remain distinctive in their …

Learning how to interact with objects is an important step towards embodied visual
intelligence, but existing techniques suffer from heavy supervision or sensing requirements …

Imitation learning is an effective and safe technique to train robot policies in the real world
because it does not depend on an expensive random exploration process. However, due to …