Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods
With extensive pretrained knowledge and high-level general capabilities, large language
models (LLMs) emerge as a promising avenue to augment reinforcement learning (RL) in …
models (LLMs) emerge as a promising avenue to augment reinforcement learning (RL) in …
Voxposer: Composable 3d value maps for robotic manipulation with language models
Large language models (LLMs) are shown to possess a wealth of actionable knowledge that
can be extracted for robot manipulation in the form of reasoning and planning. Despite the …
can be extracted for robot manipulation in the form of reasoning and planning. Despite the …
Spatialvlm: Endowing vision-language models with spatial reasoning capabilities
Understanding and reasoning about spatial relationships is crucial for Visual Question
Answering (VQA) and robotics. Vision Language Models (VLMs) have shown impressive …
Answering (VQA) and robotics. Vision Language Models (VLMs) have shown impressive …
A survey of optimization-based task and motion planning: From classical to learning approaches
Task and motion planning (TAMP) integrates high-level task planning and low-level motion
planning to equip robots with the autonomy to effectively reason over long-horizon, dynamic …
planning to equip robots with the autonomy to effectively reason over long-horizon, dynamic …
AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agents
Traditional approaches in physics-based motion generation centered around imitation
learning and reward shaping often struggle to adapt to new scenarios. To tackle this …
learning and reward shaping often struggle to adapt to new scenarios. To tackle this …
Vision-language models as a source of rewards
Building generalist agents that can accomplish many goals in rich open-ended
environments is one of the research frontiers for reinforcement learning. A key limiting factor …
environments is one of the research frontiers for reinforcement learning. A key limiting factor …
Rl-vlm-f: Reinforcement learning from vision language foundation model feedback
Reward engineering has long been a challenge in Reinforcement Learning (RL) research,
as it often requires extensive human effort and iterative processes of trial-and-error to design …
as it often requires extensive human effort and iterative processes of trial-and-error to design …
Generative ai for self-adaptive systems: State of the art and research roadmap
Self-adaptive systems (SASs) are designed to handle changes and uncertainties through a
feedback loop with four core functionalities: monitoring, analyzing, planning, and execution …
feedback loop with four core functionalities: monitoring, analyzing, planning, and execution …
FreeMotion: MoCap-Free Human Motion Synthesis with Multimodal Large Language Models
Human motion synthesis is a fundamental task in computer animation. Despite recent
progress in this field utilizing deep learning and motion capture data, existing methods are …
progress in this field utilizing deep learning and motion capture data, existing methods are …
Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
Large vision-language models (VLMs) fine-tuned on specialized visual instruction-following
data have exhibited impressive language reasoning capabilities across various scenarios …
data have exhibited impressive language reasoning capabilities across various scenarios …