Abstract We present Fast Language-Image Pre-training (FLIP), a simple and more efficient
method for training CLIP. Our method randomly masks out and removes a large portion of …

Abstract Contrastive Language-Image Pre-training (CLIP) has significantly boosted the
performance of various vision-language tasks by scaling up the dataset with image-text pairs …

Vision-language models such as CLIP learn a generic text-image embedding from large-
scale training data. A vision-language model can be adapted to a new classification task …

We propose a simple pairwise sigmoid loss for image-text pre-training. Unlike standard
contrastive learning with softmax normalization, the sigmoid loss operates solely on image …

Data augmentation is a necessity to enhance data efficiency in deep learning. For vision-
language pre-training, data is only augmented either for images or for text in previous works …

Large vision-language models are generally applicable to many downstream tasks, but
come at an exorbitant training cost that only large institutions can afford. This paper trades …

Abstract Contrastive Language-Image Pretraining has emerged as a prominent approach for
training vision and text encoders with uncurated image-text pairs from the web. To enhance …

Abstract Contrastive Language-Image Pre-training (CLIP) stands as one of the most effective
and scalable methods for training transferable vision models using paired image and text …

In vision-language modeling, image token removal is an efficient augmentation technique to
reduce the cost of encoding image features. The CLIP-style models, however, have been …

Contrastive Language-Image Pretraining (CLIP) has emerged as a novel paradigm to learn
visual models from language supervision. While researchers continue to push the frontier of …