Contrastive representation learning: A framework and review

PH Le-Khac, G Healy, AF Smeaton - Ieee Access, 2020 - ieeexplore.ieee.org
Contrastive Learning has recently received interest due to its success in self-supervised
representation learning in the computer vision domain. However, the origins of Contrastive …

Why should we add early exits to neural networks?

S Scardapane, M Scarpiniti, E Baccarelli… - Cognitive Computation, 2020 - Springer
Deep neural networks are generally designed as a stack of differentiable layers, in which a
prediction is obtained only after running the full stack. Recently, some contributions have …

The forward-forward algorithm: Some preliminary investigations

G Hinton - arXiv preprint arXiv:2212.13345, 2022 - arxiv.org
The aim of this paper is to introduce a new learning procedure for neural networks and to
demonstrate that it works well enough on a few small problems to be worth further …

Hard negative mixing for contrastive learning

Y Kalantidis, MB Sariyildiz, N Pion… - Advances in neural …, 2020 - proceedings.neurips.cc
Contrastive learning has become a key component of self-supervised learning approaches
for computer vision. By learning to embed two augmented versions of the same image close …

Contrastive learning for unpaired image-to-image translation

T Park, AA Efros, R Zhang, JY Zhu - … , Glasgow, UK, August 23–28, 2020 …, 2020 - Springer
In image-to-image translation, each patch in the output should reflect the content of the
corresponding patch in the input, independent of domain. We propose a straightforward …

Reconsidering representation alignment for multi-view clustering

DJ Trosten, S Lokse, R Jenssen… - Proceedings of the …, 2021 - openaccess.thecvf.com
Aligning distributions of view representations is a core component of today's state of the art
models for deep multi-view clustering. However, we identify several drawbacks with naively …

Masked image modeling with local multi-scale reconstruction

H Wang, Y Tang, Y Wang, J Guo… - Proceedings of the …, 2023 - openaccess.thecvf.com
Abstract Masked Image Modeling (MIM) achieves outstanding success in self-supervised
representation learning. Unfortunately, MIM models typically have huge computational …

Greedy hierarchical variational autoencoders for large-scale video prediction

B Wu, S Nair, R Martin-Martin… - Proceedings of the …, 2021 - openaccess.thecvf.com
A video prediction model that generalizes to diverse scenes would enable intelligent agents
such as robots to perform a variety of tasks via planning with the model. However, while …

Uncovering mesa-optimization algorithms in transformers

J Von Oswald, M Schlegel, A Meulemans… - arXiv preprint arXiv …, 2023 - arxiv.org
Some autoregressive models exhibit in-context learning capabilities: being able to learn as
an input sequence is processed, without undergoing any parameter changes, and without …

Augmentation invariant and instance spreading feature for softmax embedding

M Ye, J Shen, X Zhang, PC Yuen… - IEEE transactions on …, 2020 - ieeexplore.ieee.org
Deep embedding learning plays a key role in learning discriminative feature
representations, where the visually similar samples are pulled closer and dissimilar samples …