Learning by watching: Physical imitation of manipulation skills from human videos

H Xiong, Q Li, YC Chen, H Bharadhwaj… - 2021 IEEE/RSJ …, 2021 - ieeexplore.ieee.org
Learning from visual data opens the potential to accrue a large range of manipulation
behaviors by leveraging human demonstrations without specifying each of them mathe …

Learning to generalize across long-horizon tasks from human demonstrations

A Mandlekar, D Xu, R Martín-Martín, S Savarese… - arXiv preprint arXiv …, 2020 - arxiv.org
Imitation learning is an effective and safe technique to train robot policies in the real world
because it does not depend on an expensive random exploration process. However, due to …

Language-conditioned imitation learning for robot manipulation tasks

S Stepputtis, J Campbell, M Phielipp… - Advances in …, 2020 - proceedings.neurips.cc
Imitation learning is a popular approach for teaching motor skills to robots. However, most
approaches focus on extracting policy parameters from execution traces alone (ie, motion …

Human demonstrations are generalizable knowledge for robots

G Chen, T Cui, T Zhou, Z Peng, M Hu, M Wang… - arXiv preprint arXiv …, 2023 - arxiv.org
Learning from human demonstrations is an emerging trend for designing intelligent robotic
systems. However, previous methods typically regard videos as instructions, simply dividing …

From play to policy: Conditional behavior generation from uncurated robot data

ZJ Cui, Y Wang, NMM Shafiullah, L Pinto - arXiv preprint arXiv:2210.10047, 2022 - arxiv.org
While large-scale sequence modeling from offline data has led to impressive performance
gains in natural language and image generation, directly translating such ideas to robotics …

Learning language-conditioned robot behavior from offline data and crowd-sourced annotation

S Nair, E Mitchell, K Chen… - Conference on Robot …, 2022 - proceedings.mlr.press
We study the problem of learning a range of vision-based manipulation tasks from a large
offline dataset of robot interaction. In order to accomplish this, humans need easy and …

Structured world models from human videos

R Mendonca, S Bahl, D Pathak - arXiv preprint arXiv:2308.10901, 2023 - arxiv.org
We tackle the problem of learning complex, general behaviors directly in the real world. We
propose an approach for robots to efficiently learn manipulation skills using only a handful of …

What matters in language conditioned robotic imitation learning over unstructured data

O Mees, L Hermann, W Burgard - IEEE Robotics and …, 2022 - ieeexplore.ieee.org
A long-standing goal in robotics is to build robots that can perform a wide range of daily
tasks from perceptions obtained with their onboard sensors and specified only via natural …

Human-in-the-loop task and motion planning for imitation learning

A Mandlekar, CR Garrett, D Xu… - Conference on Robot …, 2023 - proceedings.mlr.press
Imitation learning from human demonstrations can teach robots complex manipulation skills,
but is time-consuming and labor intensive. In contrast, Task and Motion Planning (TAMP) …

Scaling up and distilling down: Language-guided robot skill acquisition

H Ha, P Florence, S Song - Conference on Robot Learning, 2023 - proceedings.mlr.press
We present a framework for robot skill acquisition, which 1) efficiently scale up data
generation of language-labelled robot data and 2) effectively distills this data down into a …