Global-local self-distillation for visual representation learning

T Lebailly, T Tuytelaars - Proceedings of the IEEE/CVF …, 2023 - openaccess.thecvf.com
The downstream accuracy of self-supervised methods is tightly linked to the proxy task
solved during training and the quality of the gradients extracted from it. Richer and more …

Adaptive similarity bootstrapping for self-distillation based representation learning

T Lebailly, T Stegmüller… - Proceedings of the …, 2023 - openaccess.thecvf.com
Most self-supervised methods for representation learning leverage a cross-view consistency
objective ie, they maximize the representation similarity of a given image's augmented …

Scaling and benchmarking self-supervised visual representation learning

P Goyal, D Mahajan, A Gupta… - Proceedings of the ieee …, 2019 - openaccess.thecvf.com
Self-supervised learning aims to learn representations from the data itself without explicit
manual supervision. Existing efforts ignore a crucial aspect of self-supervised learning-the …

Self-distilled self-supervised representation learning

J Jang, S Kim, K Yoo, C Kong… - Proceedings of the …, 2023 - openaccess.thecvf.com
State-of-the-art frameworks in self-supervised learning have recently shown that fully
utilizing transformer-based models can lead to performance boost compared to conventional …

Stable and Causal Inference for Discriminative Self-supervised Deep Visual Representations

Y Yang, H Li, Y Chen - Proceedings of the IEEE/CVF …, 2023 - openaccess.thecvf.com
In recent years, discriminative self-supervised methods have made significant strides in
advancing various visual tasks. The central idea of learning a data encoder that is robust to …

Contrasting contrastive self-supervised representation learning pipelines

K Kotar, G Ilharco, L Schmidt… - Proceedings of the …, 2021 - openaccess.thecvf.com
In the past few years, we have witnessed remarkable breakthroughs in self-supervised
representation learning. Despite the success and adoption of representations learned …

Understanding dimensional collapse in contrastive self-supervised learning

L Jing, P Vincent, Y LeCun, Y Tian - arXiv preprint arXiv:2110.09348, 2021 - arxiv.org
Self-supervised visual representation learning aims to learn useful representations without
relying on human annotations. Joint embedding approach bases on maximizing the …

Equimod: An equivariance module to improve self-supervised learning

A Devillers, M Lefort - arXiv preprint arXiv:2211.01244, 2022 - arxiv.org
Self-supervised visual representation methods are closing the gap with supervised learning
performance. These methods rely on maximizing the similarity between embeddings of …

Semantic-aware auto-encoders for self-supervised representation learning

G Wang, Y Tang, L Lin… - Proceedings of the IEEE …, 2022 - openaccess.thecvf.com
The resurgence of unsupervised learning can be attributed to the remarkable progress of
self-supervised learning, which includes generative (G) and discriminative (D) models. In …

Can Generative Models Improve Self-Supervised Representation Learning?

A Afkanpour, VR Khazaie, S Ayromlou… - arXiv preprint arXiv …, 2024 - arxiv.org
The rapid advancement in self-supervised learning (SSL) has highlighted its potential to
leverage unlabeled data for learning powerful visual representations. However, existing SSL …