Towards Generalist Robot Learning from Internet Video: A Survey

R McCarthy, DCH Tan, D Schmidt, F Acero… - arXiv preprint arXiv …, 2024 - arxiv.org
This survey presents an overview of methods for learning from video (LfV) in the context of
reinforcement learning (RL) and robotics. We focus on methods capable of scaling to large …

Learning Manipulation by Predicting Interaction

J Zeng, Q Bu, B Wang, W Xia, L Chen, H Dong… - arXiv preprint arXiv …, 2024 - arxiv.org
Representation learning approaches for robotic manipulation have boomed in recent years.
Due to the scarcity of in-domain robot data, prevailing methodologies tend to leverage large …

Efficient Data Collection for Robotic Manipulation via Compositional Generalization

J Gao, A Xie, T Xiao, C Finn, D Sadigh - arXiv preprint arXiv:2403.05110, 2024 - arxiv.org
Data collection has become an increasingly important problem in robotic manipulation, yet
there still lacks much understanding of how to effectively collect data to facilitate broad …

RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches

P Sundaresan, Q Vuong, J Gu, P Xu, T Xiao, S Kirmani… - 2024 - openreview.net
Natural language and images are commonly used as goal representations in goal-
conditioned imitation learning (IL). However, natural language can be ambiguous and …