In this paper, we study how to use masked signal modeling in vision and language (V+ L) representation learning. Instead of developing masked language modeling (MLM) and …
We launch EVA, a vision-centric foundation model to explore the limits of visual representation at scale using only publicly accessible data. EVA is a vanilla ViT pre-trained …
Multimodal representation learning has shown promising improvements on various vision- language tasks. Most existing methods excel at building global-level alignment between …
Vision-language representation learning largely benefits from image-text alignment through contrastive losses (eg, InfoNCE loss). The success of this alignment strategy is attributed to …
Large-scale pre-training methods of learning cross-modal representations on image-text pairs are becoming popular for vision-language tasks. While existing methods simply …
Z Lin, E Bas, KY Singh… - Proceedings of the …, 2023 - openaccess.thecvf.com
Multimodal representation learning for images with paired raw texts can improve the usability and generality of the learned semantic concepts while significantly reducing …
Building scalable models to learn from diverse, multimodal data remains an open challenge. For vision-language data, the dominant approaches are based on contrastive learning …
Contrastive learning has emerged as an efficient framework to learn multimodal representations. CLIP, a seminal work in this area, achieved impressive results by training …