Pretrained Foundation Models (PFMs) are regarded as the foundation for various
downstream tasks with different data modalities. A PFM (eg, BERT, ChatGPT, and GPT-4) is …

In recent years, neural architecture-based recommender systems have achieved
tremendous success, but they still fall short of expectation when dealing with highly sparse …

Prevalent semantic segmentation solutions, despite their different network designs (FCN
based or attention based) and mask decoding strategies (parametric softmax based or pixel …

We present a novel masked image modeling (MIM) approach, context autoencoder (CAE),
for self-supervised representation pretraining. We pretrain an encoder by making predictions …

Recently, graph collaborative filtering methods have been proposed as an effective
recommendation approach, which can capture users' preference over items by modeling the …

Real-world data typically follow a long-tailed distribution, where a few majority categories
occupy most of the data while most minority categories contain a limited number of samples …

We introduce a self-supervised vision representation model BEiT, which stands for
Bidirectional Encoder representation from Image Transformers. Following BERT developed …

Learning with few labeled data has been a longstanding problem in the computer vision and
machine learning research community. In this paper, we introduced a new semi-supervised …

In this paper, we question if self-supervised learning provides new properties to Vision
Transformer (ViT) that stand out compared to convolutional networks (convnets). Beyond the …

Pre-trained representations are becoming crucial for many NLP and perception tasks. While
representation learning in NLP has transitioned to training on raw text without human …