Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking

H Bharadhwaj, J Vakil, M Sharma, A Gupta… - arXiv preprint arXiv …, 2023 - arxiv.org
The grand aim of having a single robot that can manipulate arbitrary objects in diverse
settings is at odds with the paucity of robotics datasets. Acquiring and growing such datasets …

Manipulate by seeing: Creating manipulation controllers from pre-trained representations

J Wang, S Dasari, MK Srirama… - Proceedings of the …, 2023 - openaccess.thecvf.com
The field of visual representation learning has seen explosive growth in the past years, but
its benefits in robotics have been surprisingly limited so far. Prior work uses generic visual …

Robocat: A self-improving foundation agent for robotic manipulation

K Bousmalis, G Vezzani, D Rao, C Devin… - arXiv preprint arXiv …, 2023 - arxiv.org
The ability to leverage heterogeneous robotic experience from different robots and tasks to
quickly master novel skills and embodiments has the potential to transform robot learning …

[PDF][PDF] Vima: General robot manipulation with multimodal prompts

Y Jiang, A Gupta, Z Zhang, G Wang… - arXiv preprint …, 2022 - authors.library.caltech.edu
Prompt-based learning has emerged as a successful paradigm in natural language
processing, where a single general-purpose language model can be instructed to perform …

Octo: An open-source generalist robot policy

OM Team, D Ghosh, H Walke, K Pertsch… - arXiv preprint arXiv …, 2024 - arxiv.org
Large policies pretrained on diverse robot datasets have the potential to transform robotic
learning: instead of training new policies from scratch, such generalist robot policies may be …

Programmatically grounded, compositionally generalizable robotic manipulation

R Wang, J Mao, J Hsu, H Zhao, J Wu, Y Gao - arXiv preprint arXiv …, 2023 - arxiv.org
Robots operating in the real world require both rich manipulation skills as well as the ability
to semantically reason about when to apply those skills. Towards this goal, recent works …

Robocat: A self-improving generalist agent for robotic manipulation

K Bousmalis, G Vezzani, D Rao, CM Devin… - … on Machine Learning …, 2023 - openreview.net
The ability to leverage heterogeneous robotic experience from different robots and tasks to
quickly master novel skills and embodiments has the potential to transform robot learning …

Instruct2act: Mapping multi-modality instructions to robotic actions with large language model

S Huang, Z Jiang, H Dong, Y Qiao, P Gao… - arXiv preprint arXiv …, 2023 - arxiv.org
Foundation models have made significant strides in various applications, including text-to-
image generation, panoptic segmentation, and natural language processing. This paper …

Droid: A large-scale in-the-wild robot manipulation dataset

A Khazatsky, K Pertsch, S Nair, A Balakrishna… - arXiv preprint arXiv …, 2024 - arxiv.org
The creation of large, diverse, high-quality robot manipulation datasets is an important
stepping stone on the path toward more capable and robust robotic manipulation policies …

Mt-opt: Continuous multi-task robotic reinforcement learning at scale

D Kalashnikov, J Varley, Y Chebotar… - arXiv preprint arXiv …, 2021 - arxiv.org
General-purpose robotic systems must master a large repertoire of diverse skills to be useful
in a range of daily tasks. While reinforcement learning provides a powerful framework for …