Vision-language pre-training: Basics, recent advances, and future trends
This monograph surveys vision-language pre-training (VLP) methods for multimodal
intelligence that have been developed in the last few years. We group these approaches …
intelligence that have been developed in the last few years. We group these approaches …
Multimodal learning with transformers: A survey
Transformer is a promising neural network learner, and has achieved great success in
various machine learning tasks. Thanks to the recent prevalence of multimodal applications …
various machine learning tasks. Thanks to the recent prevalence of multimodal applications …
Scaling up visual and vision-language representation learning with noisy text supervision
Pre-trained representations are becoming crucial for many NLP and perception tasks. While
representation learning in NLP has transitioned to training on raw text without human …
representation learning in NLP has transitioned to training on raw text without human …
Violet: End-to-end video-language transformers with masked visual-token modeling
A great challenge in video-language (VidL) modeling lies in the disconnection between
fixed video representations extracted from image/video understanding models and …
fixed video representations extracted from image/video understanding models and …
Med-unic: Unifying cross-lingual medical vision-language pre-training by diminishing bias
The scarcity of data presents a critical obstacle to the efficacy of medical vision-language pre-
training (VLP). A potential solution lies in the combination of datasets from various language …
training (VLP). A potential solution lies in the combination of datasets from various language …
Deep vision multimodal learning: Methodology, benchmark, and trend
Deep vision multimodal learning aims at combining deep visual representation learning with
other modalities, such as text, sound, and data collected from other sensors. With the fast …
other modalities, such as text, sound, and data collected from other sensors. With the fast …
Wukong: A 100 million large-scale chinese cross-modal pre-training benchmark
Abstract Vision-Language Pre-training (VLP) models have shown remarkable performance
on various downstream tasks. Their success heavily relies on the scale of pre-trained cross …
on various downstream tasks. Their success heavily relies on the scale of pre-trained cross …
Altclip: Altering the language encoder in clip for extended language capabilities
In this work, we present a conceptually simple and effective method to train a strong
bilingual/multilingual multimodal representation model. Starting from the pre-trained …
bilingual/multilingual multimodal representation model. Starting from the pre-trained …
Transformers in speech processing: A survey
The remarkable success of transformers in the field of natural language processing has
sparked the interest of the speech-processing community, leading to an exploration of their …
sparked the interest of the speech-processing community, leading to an exploration of their …
Class-agnostic object detection with multi-modal transformer
What constitutes an object? This has been a long-standing question in computer vision.
Towards this goal, numerous learning-free and learning-based approaches have been …
Towards this goal, numerous learning-free and learning-based approaches have been …