Vision and text have been fully explored in contemporary video-text foundational models,
while other modalities such as audio and subtitles in videos have not received sufficient …

In this paper, we propose a Vision-Audio-Language Omni-peRception pretraining model
(VALOR) for multi-modal understanding and generation. Different from widely-studied vision …

Abstract Leveraging Large Language Models' remarkable proficiency in text-based tasks,
recent works on Multi-modal LLMs (MLLMs) extend them to other modalities like vision and …

In this paper, we propose the Vision-Audio-Language Omni-peRception pretraining model
(VALOR) for multimodal understanding and generation. Unlike widely-studied vision …

Video description generates natural language sentences that describe the subject, verb, and
objects of the targeted Video. The video description has been used to help visually impaired …

Video-based commonsense captioning aims to generate captions for the video content
while providing multiple commonsense about the underlying event. Existing methods utilize …

Abstract The Temporal Video Scene Segmentation (TVSS) task is still an open problem
presenting challenges in the Multimedia Analysis area. Current approaches employ …

Diffusion-based models have recently demonstrated notable success in various generative
tasks involving continuous signals, such as image, video, and audio synthesis. However …