Robot learning in the era of foundation models: A survey

X Xiao, J Liu, Z Wang, Y Zhou, Y Qi, Q Cheng… - arXiv preprint arXiv …, 2023 - arxiv.org
The proliferation of Large Language Models (LLMs) has s fueled a shift in robot learning
from automation towards general embodied Artificial Intelligence (AI). Adopting foundation …

Real-World Robot Applications of Foundation Models: A Review

K Kawaharazuka, T Matsushima… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent developments in foundation models, like Large Language Models (LLMs) and Vision-
Language Models (VLMs), trained on extensive data, facilitate flexible application across …

RoboCoder: Robotic Learning from Basic Skills to General Tasks with Large Language Models

J Li, P Chen, S Wu, C Zheng, H Xu, J Jia - arXiv preprint arXiv:2406.03757, 2024 - arxiv.org
The emergence of Large Language Models (LLMs) has improved the prospects for robotic
tasks. However, existing benchmarks are still limited to single tasks with limited …

A Survey on Robotics with Foundation Models: toward Embodied AI

Z Xu, K Wu, J Wen, J Li, N Liu, Z Che, J Tang - arXiv preprint arXiv …, 2024 - arxiv.org
While the exploration for embodied AI has spanned multiple decades, it remains a persistent
challenge to endow agents with human-level intelligence, including perception, learning …

Toward general-purpose robots via foundation models: A survey and meta-analysis

Y Hu, Q Xie, V Jain, J Francis, J Patrikar… - arXiv preprint arXiv …, 2023 - arxiv.org
Building general-purpose robots that can operate seamlessly, in any environment, with any
object, and utilizing various skills to complete diverse tasks has been a long-standing goal in …

Foundation models in robotics: Applications, challenges, and the future

R Firoozi, J Tucker, S Tian, A Majumdar, J Sun… - arXiv preprint arXiv …, 2023 - arxiv.org
We survey applications of pretrained foundation models in robotics. Traditional deep
learning models in robotics are trained on small datasets tailored for specific tasks, which …

Neural Scaling Laws for Embodied AI

S Sartor, N Thompson - arXiv preprint arXiv:2405.14005, 2024 - arxiv.org
Scaling laws have driven remarkable progress across machine learning domains like
language modeling and computer vision. However, the exploration of scaling laws in …

MOTO: Offline pre-training to online fine-tuning for model-based robot learning

R Rafailov, KB Hatch, V Kolev… - … on Robot Learning, 2023 - proceedings.mlr.press
We study the problem of offline pre-training and online fine-tuning for reinforcement learning
from high-dimensional observations in the context of realistic robot tasks. Recent offline …

Robogen: Towards unleashing infinite data for automated robot learning via generative simulation

Y Wang, Z Xian, F Chen, TH Wang, Y Wang… - arXiv preprint arXiv …, 2023 - arxiv.org
We present RoboGen, a generative robotic agent that automatically learns diverse robotic
skills at scale via generative simulation. RoboGen leverages the latest advancements in …

Vision-language foundation models as effective robot imitators

X Li, M Liu, H Zhang, C Yu, J Xu, H Wu… - arXiv preprint arXiv …, 2023 - arxiv.org
Recent progress in vision language foundation models has shown their ability to understand
multimodal data and resolve complicated vision language tasks, including robotics …