We propose a new task to benchmark scene understanding of embodied agents: Situated
Question Answering in 3D Scenes (SQA3D). Given a scene context (eg, 3D scan), SQA3D …

3D dense captioning aims to semantically describe each object detected in a 3D scene,
which plays a significant role in 3D scene understanding. Previous works lack a complete …

3D dense captioning stands as a cornerstone in achieving a comprehensive understanding
of 3D scenes through natural language. It has recently witnessed remarkable achievements …

The fusion of multi-modal perception in autonomous driving plays a pivotal role in vehicle
behavior decision-making. However, much of the previous research has predominantly …

Recently, 3D understanding has become popular to facilitate autonomous agents to perform
further decisionmaking. However, existing 3D datasets and methods are often limited to …

We introduce Point-Bind, a 3D multi-modality model aligning point clouds with 2D image,
language, audio, and video. Guided by ImageBind, we construct a joint embedding space …

Referring expression comprehension (REC) and segmentation (RES) are two highly-related
tasks, which both aim at identifying the referent according to a natural language expression …

In the field of autonomous vehicles (AVs), accurately discerning commander intent and
executing linguistic commands within a visual context presents a significant challenge. This …

Enabling Large Language Models (LLMs) to interact with 3D environments is challenging.
Existing approaches extract point clouds either from ground truth (GT) geometry or 3D …

Spoken language understanding (SLU) systems aim to understand users' utterance, which
is a key component of task-oriented dialogue systems. In this paper, we focus on improving …