Abstract We present Fast Language-Image Pre-training (FLIP), a simple and more efficient
method for training CLIP. Our method randomly masks out and removes a large portion of …

Masked image modeling (MIM) has achieved promising results on various vision tasks.
However, the limited discriminability of learned representation manifests there is still plenty …

Transformers and masked language modeling are quickly being adopted and explored in
computer vision as vision transformers and masked image modeling (MIM). In this work, we …

Masked image modeling (MIM) has attracted much research attention due to its promising
potential for learning scalable visual representations. In typical approaches, models usually …

Self-supervised pretraining bears the potential to generate expressive representations from
large-scale Earth observation (EO) data without human annotation. However, most existing …

As a pioneering work, PointContrast conducts unsupervised 3D representation learning via
leveraging contrastive learning over raw RGB-D frames and proves its effectiveness on …

As the deep learning revolution marches on, self-supervised learning has garnered
increasing attention in recent years thanks to its remarkable representation learning ability …

Masked autoencoder (MAE), a simple and effective self-supervised learning framework
based on the reconstruction of masked image regions, has recently achieved prominent …

Masked image modeling (MIM) learns visual representation by masking and reconstructing
image patches. Applying the reconstruction supervision on the CLIP representation has …

Self-supervised representation learning follows a paradigm of withholding some part of the
data and tasking the network to predict it from the remaining part. Among many techniques …