A survey on robotics with foundation models: toward embodied ai

Z Xu, K Wu, J Wen, J Li, N Liu, Z Che, J Tang - arXiv preprint arXiv …, 2024 - arxiv.org
While the exploration for embodied AI has spanned multiple decades, it remains a persistent
challenge to endow agents with human-level intelligence, including perception, learning …

Toward general-purpose robots via foundation models: A survey and meta-analysis

Y Hu, Q Xie, V Jain, J Francis, J Patrikar… - arXiv preprint arXiv …, 2023 - arxiv.org
Building general-purpose robots that can operate seamlessly, in any environment, with any
object, and utilizing various skills to complete diverse tasks has been a long-standing goal in …

Foundation models in robotics: Applications, challenges, and the future

R Firoozi, J Tucker, S Tian, A Majumdar, J Sun… - arXiv preprint arXiv …, 2023 - arxiv.org
We survey applications of pretrained foundation models in robotics. Traditional deep
learning models in robotics are trained on small datasets tailored for specific tasks, which …

What Foundation Models can Bring for Robot Learning in Manipulation: A Survey

D Li, Y Jin, H Yu, J Shi, X Hao, P Hao, H Liu… - arXiv preprint arXiv …, 2024 - arxiv.org
The realization of universal robots is an ultimate goal of researchers. However, a key hurdle
in achieving this goal lies in the robots' ability to manipulate objects in their unstructured …

Position Paper: Agent AI Towards a Holistic Intelligence

Q Huang, N Wake, B Sarkar, Z Durante, R Gong… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent advancements in large foundation models have remarkably enhanced our
understanding of sensory information in open-world environments. In leveraging the power …

Towards generalist robots: A promising paradigm via generative simulation

Z Xian, T Gervet, Z Xu, YL Qiao, TH Wang… - arXiv preprint arXiv …, 2023 - arxiv.org
This document serves as a position paper that outlines the authors' vision for a potential
pathway towards generalist robots. The purpose of this document is to share the excitement …

Allenact: A framework for embodied ai research

L Weihs, J Salvador, K Kotar, U Jain, KH Zeng… - arXiv preprint arXiv …, 2020 - arxiv.org
The domain of Embodied AI, in which agents learn to complete tasks through interaction with
their environment from egocentric observations, has experienced substantial growth with the …

ASC me to do anything: multi-task training for embodied ai

J Lu, J Salvador, R Mottaghi, A Kembhavi - arXiv preprint arXiv …, 2022 - arxiv.org
Embodied AI has seen steady progress across a diverse set of independent tasks. While
these varied tasks have different end goals, the basic skills required to complete them …

🏘️ ProcTHOR: Large-Scale Embodied AI Using Procedural Generation

M Deitke, E VanderBilt, A Herrasti… - Advances in …, 2022 - proceedings.neurips.cc
Massive datasets and high-capacity models have driven many recent advancements in
computer vision and natural language understanding. This work presents a platform to …

Robot learning in the era of foundation models: A survey

X Xiao, J Liu, Z Wang, Y Zhou, Y Qi, Q Cheng… - arXiv preprint arXiv …, 2023 - arxiv.org
The proliferation of Large Language Models (LLMs) has s fueled a shift in robot learning
from automation towards general embodied Artificial Intelligence (AI). Adopting foundation …