… representation learning and design a mask contrast (MaskCo) framework to implement the
… gap between masked and unmasked features, we design a dedicated mask prediction head …

… visual knowledge about the visible structures. In this work, we show that predicting certain
masked … A simple framework for contrastive learning of visual representations. In ICML, 2020. …

… to learn good representations for action recognition [49… , we apply masked visual modeling
for video prediction, and … line of work is leveraging good visual representations learnt via self …

… speech, which masks multi-stream video input and predicts … AV-HuBERT learns powerful
audio-visual speech representation … Using our audio-visual representation on the same …

… are randomly masked, and the objective is to predict the original information of the masked
… In order to avoid masking the tokens of crucial region, we propose a masked token strategy …

… prediction task for the autoencoder. Specifically, we separately update visual representations
… masking and reward prediction, and (ii) learning the latent dynamics model that predicts …

… limits of large scale MIM pre-training via masked image-text aligned feature prediction [43…
masked visual representation learning. We show simple masked feature modeling as a visual …

… Compared to MLM, we propose to predict masked tokens in a caption by using the visual
information computed by \(\phi \). This way, we learn visual representations that should be …

… of masked prediction, also known as masked autoencoding, … This task of masked prediction
not only forces the model to … the combination of masked prediction and transformer sequence …

… language modeling as a pretext task for self-supervised visual representation learning. …
representations learnt by a masked prediction task (our approach), and a joint masked prediction …