Our experience of the world is multimodal-we see objects, hear sounds, feel texture, smell
odors, and taste flavors. Modality refers to the way in which something happens or is …

Abstract Automatic Visual Captioning (AVC) generates syntactically and semantically correct
sentences by describing important objects, attributes, and their relationships with each other …

Combining complementary information from multiple modalities is intuitively appealing for
improving the performance of learning-based approaches. However, it is challenging to fully …

Environmental Sound Classification (ESC) is an active research area in the audio domain
and has seen a lot of progress in the past years. However, many of the existing approaches …

Describing videos with natural language is one of the ultimate goals of video understanding.
Video records multi-modal information including image, motion, aural, speech and so on …

Crowd estimation is a very challenging problem. The most recent study tries to exploit
auditory information to aid the visual models, however, the performance is limited due to the …

The topic diversity of open-domain videos leads to various vocabularies and linguistic
expressions in describing video contents, and therefore, makes the video captioning task …

This paper focuses on an important problem of detecting offensive analogy meme on online
social media where the visual content and the texts/captions of the meme together make an …

Multiple modalities can provide more valuable information than single one by describing the
same contents in various ways. Previous methods mainly focus on fusing the shallow …

Automatic video description generation (aka video captioning) is one of the ultimate goals
for video understanding. Despite the wide range of applications such as video indexing and …