Sight and hearing are two senses that play a vital role in human communication and scene
understanding. To mimic human perception ability, audio-visual learning, aimed at …

Salient object detection (SOD) is a long-standing research topic in computer vision with
increasing interest in the past decade. Since light fields record comprehensive information of …

Multi-modal learning from video data has seen increased attention recently as it allows
training of semantically meaningful embeddings without human annotation, enabling tasks …

Although video summarization has achieved tremendous success benefiting from Recurrent
Neural Networks (RNN), RNN-based methods neglect the global dependencies and multi …

We propose the ViNet architecture for audio-visual saliency prediction. ViNet is a fully
convolutional encoder-decoder architecture. The encoder uses visual features from a …

Incorporating the audio stream enables Video Saliency Prediction (VSP) to imitate the
selective attention mechanism of human brain. By focusing on the benefits of joint auditory …

Predicting human's gaze from egocentric videos serves as a critical role for human intention
understanding in daily activities. In this paper, we present the first transformer-based model …

Thanks to the rapid advances in the deep learning techniques and the wide availability of
large-scale training sets, the performances of video saliency detection models have been …

This paper strives for repetitive activity counting in videos. Different from existing works,
which all analyze the visual video content only, we incorporate for the first time the …

We address the problem of estimating depth with multi modal audio visual data. Inspired by
the ability of animals, such as bats and dolphins, to infer distance of objects with …