This paper introduces a novel task called Cross Modal Generalization (CMG), which
addresses the challenge of learning a unified discrete representation from paired …

Deep learning has achieved notable success across academia, medicine, and industry. Its
ability to identify complex patterns in large-scale data and to manage millions of parameters …

Cross-modality transfer aims to leverage large pretrained models to complete tasks that may
not belong to the modality of pretraining data. Existing works achieve certain success in …

Video self-supervised learning (VSSL) has made significant progress in recent years.
However, the exact behavior and dynamics of these models under different forms of …

Modality gap between RGB and thermal infrared (TIR) images is a crucial issue but often
overlooked in existing RGBT tracking methods. It can be observed that modality gap mainly …

Audio-visual correlation learning aims to capture and understand natural phenomena
between audio and visual data. The rapid growth of Deep Learning propelled the …

Human emotion is a complex phenomenon conveyed and perceived through facial
expressions, vocal tones, body language, and physiological signals. Multimodal emotion …

Cross-modal knowledge distillation (CMKD) refers to the scenario in which a learning
framework must handle training and test data that exhibit a modality mismatch, more …

Recent advancements in self-supervised audio-visual representation learning have
demonstrated its potential to capture rich and comprehensive representations. However …

Incorporating domain-specific visual information into text poses one of the critical challenges
for domain-specific multi-modal neural machine translation (DMNMT). While most existing …