Distilling large vision-language model with out-of-distribution generalizability

X Li, Y Fang, M Liu, Z Ling, Z Tu… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
Large vision-language models have achieved outstanding performance, but their size and
computational requirements make their deployment on resource-constrained devices and …

Learning environment-aware affordance for 3d articulated object manipulation under occlusions

R Wu, K Cheng, Y Zhao, C Ning… - Advances in Neural …, 2024 - proceedings.neurips.cc
Perceiving and manipulating 3D articulated objects in diverse environments is essential for
home-assistant robots. Recent studies have shown that point-level affordance provides …

Robot learning in the era of foundation models: A survey

X Xiao, J Liu, Z Wang, Y Zhou, Y Qi, Q Cheng… - arXiv preprint arXiv …, 2023 - arxiv.org
The proliferation of Large Language Models (LLMs) has s fueled a shift in robot learning
from automation towards general embodied Artificial Intelligence (AI). Adopting foundation …

Td-mpc2: Scalable, robust world models for continuous control

N Hansen, H Su, X Wang - arXiv preprint arXiv:2310.16828, 2023 - arxiv.org
TD-MPC is a model-based reinforcement learning (RL) algorithm that performs local
trajectory optimization in the latent space of a learned implicit (decoder-free) world model. In …

Robogen: Towards unleashing infinite data for automated robot learning via generative simulation

Y Wang, Z Xian, F Chen, TH Wang, Y Wang… - arXiv preprint arXiv …, 2023 - arxiv.org
We present RoboGen, a generative robotic agent that automatically learns diverse robotic
skills at scale via generative simulation. RoboGen leverages the latest advancements in …

Mimicgen: A data generation system for scalable robot learning using human demonstrations

A Mandlekar, S Nasiriany, B Wen, I Akinola… - arXiv preprint arXiv …, 2023 - arxiv.org
Imitation learning from a large set of human demonstrations has proved to be an effective
paradigm for building capable robot agents. However, the demonstrations can be extremely …

Orbit: A unified simulation framework for interactive robot learning environments

M Mittal, C Yu, Q Yu, J Liu, N Rudin… - IEEE Robotics and …, 2023 - ieeexplore.ieee.org
We present Orbit, a unified and modular framework for robot learning powered by Nvidia
Isaac Sim. It offers a modular design to easily and efficiently create robotic environments …

Rl-vigen: A reinforcement learning benchmark for visual generalization

Z Yuan, S Yang, P Hua, C Chang… - Advances in Neural …, 2024 - proceedings.neurips.cc
Abstract Visual Reinforcement Learning (Visual RL), coupled with high-dimensional
observations, has consistently confronted the long-standing challenge of out-of-distribution …

Partmanip: Learning cross-category generalizable part manipulation policy from point cloud observations

H Geng, Z Li, Y Geng, J Chen… - Proceedings of the …, 2023 - openaccess.thecvf.com
Learning a generalizable object manipulation policy is vital for an embodied agent to work in
complex real-world scenes. Parts, as the shared components in different object categories …

Dexart: Benchmarking generalizable dexterous manipulation with articulated objects

C Bao, H Xu, Y Qin, X Wang - Proceedings of the IEEE/CVF …, 2023 - openaccess.thecvf.com
To enable general-purpose robots, we will require the robot to operate daily articulated
objects as humans do. Current robot manipulation has heavily relied on using a parallel …