Moving around in the world is naturally a multisensory experience, but today's embodied
agents are deaf—restricted to solely their visual perception of the environment. We introduce …

How much can we infer about a person's looks from the way they speak? In this paper, we
study the task of reconstructing a facial image of a person from a short audio recording of …

Several animal species (eg, bats, dolphins, and whales) and even visually impaired humans
have the remarkable ability to perform echolocation: a biological sonar used to perceive …

Speaker diarization consists of assigning speech signals to people engaged in a dialogue.
An audio-visual spatiotemporal diarization model is proposed. The model is well suited for …

In this paper, we study the associations between human faces and voices. Audiovisual
integration, specifically the integration of facial and vocal information is a well-researched …

Sequential Monte Carlo probability hypothesis density (SMC-PHD) filtering is a popular
method used recently for audio-visual (AV) multi-speaker tracking. However, due to the …

Compact multi-sensor platforms are portable and thus desirable for robotics and personal-
assistance tasks. However, compared to physically distributed sensors, the size of these …

The ability for a smart speaker to localize a user based on his/her voice opens the door to
many new applications. In this paper, we present a novel system, MAVL, to localize human …

We present an automatic voice activity detection (VAD) method that is solely based on visual
cues. Unlike traditional approaches processing audio, we show that upper body motion …

In this paper, we present a system that associates faces with voices in a video by fusing
information from the audio and visual signals. The thesis underlying our work is that an …