If generalist robots are to operate in truly unstructured environments, they need to be able to
recognize and reason about novel objects and scenarios. Such objects and scenarios might …

Many problems in image processing and computer vision (eg colorization, style transfer) can
be posed as' manipulating'an input image into a corresponding output image given a user …

For robots to follow instructions from people, they must be able to connect the rich semantic
information in human vocabulary, eg" can you get me the pink stuffed whale?" to their …

Robot learning methods have the potential for widespread generalization across tasks,
environments, and objects. However, these methods require large diverse datasets that are …

Generative pre-trained models have demonstrated remarkable effectiveness in language
and vision domains by learning useful representations. In this paper, we extend the scope of …

Can we learn robot manipulation for everyday tasks, only by watching videos of humans
doing arbitrary tasks in different unstructured settings? Unlike widely adopted strategies of …

Does having visual priors (eg the ability to detect objects) facilitate learning to perform vision-
based manipulation (eg picking up objects)? We study this problem under the framework of …

Diffusion models generate images by iterative denoising. Recent work has shown that by
making the denoising process deterministic, one can encode real images into latent codes …

Text-driven image manipulation remains challenging in training or inference flexibility.
Conditional generative models depend heavily on expensive annotated training data …

Generalization has been a long-standing challenge for reinforcement learning (RL). Visual
RL, in particular, can be easily distracted by irrelevant factors in high-dimensional …