Large language models (LLMs) trained on code-completion have been shown to be capable
of synthesizing simple Python programs from docstrings [1]. We find that these code-writing …

Grounding language to the visual observations of a navigating agent can be performed
using off-the-shelf visual-language models pretrained on Internet-scale data (eg, image …

Imitation learning aims to extract high-performance policies from logged demonstrations of
expert behavior. It is common to frame imitation learning as a supervised learning problem …

We study the choice of action space in robot manipulation learning and sim-to-real transfer.
We define metrics that assess the performance, and examine the emerging properties in the …

Robot learning provides a number of ways to teach robots simple skills, such as grasping.
However, these skills are usually trained in open, clutter-free environments, and therefore …

Imitation learning addresses the challenge of learning by observing an expert's
demonstrations without access to reward signals from environments. Most existing imitation …

High-level cognitive assistance, such as predicting dissection trajectories in Endoscopic
Submucosal Dissection (ESD), can potentially support and facilitate surgical skills training …

Arranging objects correctly is a key capability for robots which unlocks a wide range of
useful tasks. A prerequisite for creating successful arrangements is the ability to evaluate the …

We have developed an autonomous robot motion generation model based on distributed
and integrated multimodal learning. Since each modality used as a robot's senses, such as …

Joint space and task space control are the two dominant action modes for controlling robot
arms within the robot learning literature. Actions in joint space provide precise control over …