Abstract We introduce SoundSpaces 2.0, a platform for on-the-fly geometry-based audio
rendering for 3D environments. Given a 3D mesh of a real-world environment …

Recent work on audio-visual navigation assumes a constantly-sounding target and restricts
the role of audio to signaling the target's position. We introduce semantic audio-visual …

The majority of prior monocular depth estimation methods without groundtruth depth
guidance focus on driving scenarios. We show that such methods generalize poorly to …

Room impulse response (RIR) functions capture how the surrounding physical environment
transforms the sounds heard by a listener, with implications for various applications in AR …

People navigating in unfamiliar buildings take advantage of myriad visual, spatial and
semantic cues to efficiently achieve their navigation goals. Towards equipping …

We introduce the active audio-visual source separation problem, where an agent must move
intelligently in order to better isolate the sounds coming from an object of interest in its …

Contextual information plays an important role in many computer vision tasks, such as object
detection, video action detection, image classification, etc. Recognizing a single object or …

We study the task of zero-shot vision-and-language navigation (ZS-VLN), a practical yet
challenging problem in which an agent learns to navigate following a path described by …

Given only acoustic signals without any high-level information, such as voices or sounds of
scenes/actions, how much can we infer about the behavior of humans? Unlike existing …

In this paper, we propose a Disentangled Counterfactual Learning (DCL) approach for
physical audiovisual commonsense reasoning. The task aims to infer objects' physics …