Large language models for robotics: A survey

F Zeng, W Gan, Y Wang, N Liu, PS Yu - arXiv preprint arXiv:2311.07226, 2023 - arxiv.org
The human ability to learn, generalize, and control complex manipulation tasks through multi-
modality feedback suggests a unique capability, which we refer to as dexterity intelligence …

Diffusion policy: Visuomotor policy learning via action diffusion

C Chi, Z Xu, S Feng, E Cousineau… - … Journal of Robotics …, 2023 - journals.sagepub.com
This paper introduces Diffusion Policy, a new way of generating robot behavior by
representing a robot's visuomotor policy as a conditional denoising diffusion process. We …

Open x-embodiment: Robotic learning datasets and rt-x models

A O'Neill, A Rehman, A Gupta, A Maddukuri… - arXiv preprint arXiv …, 2023 - arxiv.org
Large, high-capacity models trained on diverse datasets have shown remarkable successes
on efficiently tackling downstream applications. In domains from NLP to Computer Vision …

Scaling up and distilling down: Language-guided robot skill acquisition

H Ha, P Florence, S Song - Conference on Robot Learning, 2023 - proceedings.mlr.press
We present a framework for robot skill acquisition, which 1) efficiently scale up data
generation of language-labelled robot data and 2) effectively distills this data down into a …

Learning fine-grained bimanual manipulation with low-cost hardware

TZ Zhao, V Kumar, S Levine, C Finn - arXiv preprint arXiv:2304.13705, 2023 - arxiv.org
Fine manipulation tasks, such as threading cable ties or slotting a battery, are notoriously
difficult for robots because they require precision, careful coordination of contact forces, and …

Affordances from human videos as a versatile representation for robotics

S Bahl, R Mendonca, L Chen… - Proceedings of the …, 2023 - openaccess.thecvf.com
Building a robot that can understand and learn to interact by watching humans has inspired
several vision problems. However, despite some successful results on static datasets, it …

Q-transformer: Scalable offline reinforcement learning via autoregressive q-functions

Y Chebotar, Q Vuong, K Hausman… - … on Robot Learning, 2023 - proceedings.mlr.press
In this work, we present a scalable reinforcement learning method for training multi-task
policies from large offline datasets that can leverage both human demonstrations and …

Interactive language: Talking to robots in real time

C Lynch, A Wahid, J Tompson, T Ding… - IEEE Robotics and …, 2023 - ieeexplore.ieee.org
We present a framework for building interactive, real-time, natural language-instructable
robots in the real world, and we open source related assets (dataset, environment …

Liv: Language-image representations and rewards for robotic control

YJ Ma, V Kumar, A Zhang, O Bastani… - International …, 2023 - proceedings.mlr.press
Abstract We present Language-Image Value learning (LIV), a unified objective for vision-
language representation and reward learning from action-free videos with text annotations …

Bridgedata v2: A dataset for robot learning at scale

HR Walke, K Black, TZ Zhao, Q Vuong… - … on Robot Learning, 2023 - proceedings.mlr.press
We introduce BridgeData V2, a large and diverse dataset of robotic manipulation behaviors
designed to facilitate research in scalable robot learning. BridgeData V2 contains 53,896 …