Mm-llms: Recent advances in multimodal large language models
In the past year, MultiModal Large Language Models (MM-LLMs) have undergone
substantial advancements, augmenting off-the-shelf LLMs to support MM inputs or outputs …
substantial advancements, augmenting off-the-shelf LLMs to support MM inputs or outputs …
A survey of reasoning with foundation models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-
world settings such as negotiation, medical diagnosis, and criminal investigation. It serves …
world settings such as negotiation, medical diagnosis, and criminal investigation. It serves …
Closed-loop open-vocabulary mobile manipulation with gpt-4v
Autonomous robot navigation and manipulation in open environments require reasoning
and replanning with closed-loop feedback. We present COME-robot, the first closed-loop …
and replanning with closed-loop feedback. We present COME-robot, the first closed-loop …
Copa: General robotic manipulation through spatial constraints of parts with foundation models
Foundation models pre-trained on web-scale data are shown to encapsulate extensive
world knowledge beneficial for robotic manipulation in the form of task planning. However …
world knowledge beneficial for robotic manipulation in the form of task planning. However …
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
Vision language models (VLMs) have shown impressive capabilities across a variety of
tasks, from logical reasoning to visual understanding. This opens the door to richer …
tasks, from logical reasoning to visual understanding. This opens the door to richer …
Learning-based legged locomotion; state of the art and future perspectives
Legged locomotion holds the premise of universal mobility, a critical capability for many real-
world robotic applications. Both model-based and learning-based approaches have …
world robotic applications. Both model-based and learning-based approaches have …
Voicepilot: Harnessing LLMs as speech interfaces for physically assistive robots
Physically assistive robots present an opportunity to significantly increase the well-being
and independence of individuals with motor impairments or other forms of disability who are …
and independence of individuals with motor impairments or other forms of disability who are …
Neural Scaling Laws for Embodied AI
S Sartor, N Thompson - arXiv preprint arXiv:2405.14005, 2024 - arxiv.org
Scaling laws have driven remarkable progress across machine learning domains like
language modeling and computer vision. However, the exploration of scaling laws in …
language modeling and computer vision. However, the exploration of scaling laws in …
Recommendations for designing conversational companion robots with older adults through foundation models
Companion robots are aimed to mitigate loneliness and social isolation among older adults
by providing social and emotional support in their everyday lives. However, older adults' …
by providing social and emotional support in their everyday lives. However, older adults' …
GPT-Fabric: Folding and Smoothing Fabric by Leveraging Pre-Trained Foundation Models
V Raval, E Zhao, H Zhang, S Nikolaidis… - arXiv preprint arXiv …, 2024 - arxiv.org
Fabric manipulation has applications in folding blankets, handling patient clothing, and
protecting items with covers. It is challenging for robots to perform fabric manipulation since …
protecting items with covers. It is challenging for robots to perform fabric manipulation since …