Mobile and wearable devices have enabled numerous applications, including activity
tracking, wellness monitoring, and human–computer interaction, that measure and improve …

Multi-view learning (MVL) has attracted increasing attention and achieved great practical
success by exploiting complementary information of multiple features or modalities …

We propose the first joint audio-video generation framework that brings engaging watching
and listening experiences simultaneously, towards high-quality realistic videos. To generate …

Generating sound effects that people want is an important topic. However, there are limited
studies in this area for sound generation. In this study, we investigate generating sound …

Multi-modal learning from video data has seen increased attention recently as it allows
training of semantically meaningful embeddings without human annotation, enabling tasks …

We introduce a new approach for audio-visual speech separation. Given a video, the goal is
to extract the speech associated with a face in spite of simultaneous back-ground sounds …

We introduce PixelPlayer, a system that, by leveraging large amounts of unlabeled videos,
learns to locate image regions which produce sounds and separate the input sounds into a …

In the face of the video data deluge, today's expensive clip-level classifiers are increasingly
impractical. We propose a framework for efficient action recognition in untrimmed video that …

In this paper, we introduce a novel problem of audio-visual event localization in
unconstrained videos. We define an audio-visual event as an event that is both visible and …

Dancing to music is an instinctive move by humans. Learning to model the music-to-dance
generation process is, however, a challenging problem. It requires significant efforts to …