Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representations
Autonomous robotic systems capable of learning novel manipulation tasks are poised to
transform industries from manufacturing to service automation. However, modern methods …
transform industries from manufacturing to service automation. However, modern methods …
Real-World Robot Applications of Foundation Models: A Review
K Kawaharazuka, T Matsushima… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent developments in foundation models, like Large Language Models (LLMs) and Vision-
Language Models (VLMs), trained on extensive data, facilitate flexible application across …
Language Models (VLMs), trained on extensive data, facilitate flexible application across …
PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models
D Guo, Y Xiang, S Zhao, X Zhu, M Tomizuka… - arXiv preprint arXiv …, 2024 - arxiv.org
Robotic grasping is a fundamental aspect of robot functionality, defining how robots interact
with objects. Despite substantial progress, its generalizability to counter-intuitive or long …
with objects. Despite substantial progress, its generalizability to counter-intuitive or long …
What Foundation Models can Bring for Robot Learning in Manipulation: A Survey
D Li, Y Jin, H Yu, J Shi, X Hao, P Hao, H Liu… - arXiv preprint arXiv …, 2024 - arxiv.org
The realization of universal robots is an ultimate goal of researchers. However, a key hurdle
in achieving this goal lies in the robots' ability to manipulate objects in their unstructured …
in achieving this goal lies in the robots' ability to manipulate objects in their unstructured …
3D Diffuser Actor: Policy Diffusion with 3D Scene Representations
We marry diffusion policies and 3D scene representations for robot manipulation. Diffusion
policies learn the action distribution conditioned on the robot and environment state using …
policies learn the action distribution conditioned on the robot and environment state using …
Understanding Long Videos in One Multimodal Language Model Pass
Large Language Models (LLMs), known to contain a strong awareness of world knowledge,
have allowed recent approaches to achieve excellent performance on Long-Video …
have allowed recent approaches to achieve excellent performance on Long-Video …
" Task Success" is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviors
Large-scale generative models are shown to be useful for sampling meaningful candidate
solutions, yet they often overlook task constraints and user preferences. Their full power is …
solutions, yet they often overlook task constraints and user preferences. Their full power is …
RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model
Robots powered by'blackbox'models need to provide human-understandable explanations
which we can trust. Hence, explainability plays a critical role in trustworthy autonomous …
which we can trust. Hence, explainability plays a critical role in trustworthy autonomous …
General Flow as Foundation Affordance for Scalable Robot Learning
We address the challenge of acquiring real-world manipulation skills with a scalable
framework. Inspired by the success of large-scale auto-regressive prediction in Large …
framework. Inspired by the success of large-scale auto-regressive prediction in Large …
A Survey of Robotic Language Grounding: Tradeoffs Between Symbols and Embeddings
With large language models, robots can understand language more flexibly and more
capable than ever before. This survey reviews recent literature and situates it into a …
capable than ever before. This survey reviews recent literature and situates it into a …