Large language models for robotics: Opportunities, challenges, and perspectives
Large language models (LLMs) have undergone significant expansion and have been
increasingly integrated across various domains. Notably, in the realm of robot task planning …
increasingly integrated across various domains. Notably, in the realm of robot task planning …
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision Language Audio and Action
We present Unified-IO 2 a multimodal and multi-skill unified model capable of following
novel instructions. Unified-IO 2 can use text images audio and/or videos as input and can …
novel instructions. Unified-IO 2 can use text images audio and/or videos as input and can …
Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation
Imitation learning from human demonstrations has shown impressive performance in
robotics. However, most results focus on table-top manipulation, lacking the mobility and …
robotics. However, most results focus on table-top manipulation, lacking the mobility and …
Real-world robot applications of foundation models: A review
K Kawaharazuka, T Matsushima… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent developments in foundation models, like Large Language Models (LLMs) and Vision-
Language Models (VLMs), trained on extensive data, facilitate flexible application across …
Language Models (VLMs), trained on extensive data, facilitate flexible application across …
Octo: An open-source generalist robot policy
Large policies pretrained on diverse robot datasets have the potential to transform robotic
learning: instead of training new policies from scratch, such generalist robot policies may be …
learning: instead of training new policies from scratch, such generalist robot policies may be …
The foundation model transparency index
Foundation models have rapidly permeated society, catalyzing a wave of generative AI
applications spanning enterprise and consumer-facing contexts. While the societal impact of …
applications spanning enterprise and consumer-facing contexts. While the societal impact of …
Zero-shot robotic manipulation with pretrained image-editing diffusion models
If generalist robots are to operate in truly unstructured environments, they need to be able to
recognize and reason about novel objects and scenarios. Such objects and scenarios might …
recognize and reason about novel objects and scenarios. Such objects and scenarios might …
Foundation models in robotics: Applications, challenges, and the future
We survey applications of pretrained foundation models in robotics. Traditional deep
learning models in robotics are trained on small datasets tailored for specific tasks, which …
learning models in robotics are trained on small datasets tailored for specific tasks, which …
Vision-language foundation models as effective robot imitators
Recent progress in vision language foundation models has shown their ability to understand
multimodal data and resolve complicated vision language tasks, including robotics …
multimodal data and resolve complicated vision language tasks, including robotics …
On bringing robots home
NMM Shafiullah, A Rai, H Etukuru, Y Liu, I Misra… - arXiv preprint arXiv …, 2023 - arxiv.org
Throughout history, we have successfully integrated various machines into our homes.
Dishwashers, laundry machines, stand mixers, and robot vacuums are a few recent …
Dishwashers, laundry machines, stand mixers, and robot vacuums are a few recent …