Head2toe: Utilizing intermediate representations for better transfer learning

U Evci, V Dumoulin, H Larochelle… - … on Machine Learning, 2022 - proceedings.mlr.press
Transfer-learning methods aim to improve performance in a data-scarce target domain using
a model pretrained on a data-rich source domain. A cost-efficient strategy, linear probing …

Adversarial training helps transfer learning via better representations

Z Deng, L Zhang, K Vodrahalli… - Advances in Neural …, 2021 - proceedings.neurips.cc
Transfer learning aims to leverage models pre-trained on source data to efficiently adapt to
target setting, where only limited data are available for model fine-tuning. Recent works …

[PDF][PDF] Large scale learning of general visual representations for transfer

A Kolesnikov, L Beyer, X Zhai… - arXiv preprint arXiv …, 2019 - ask.qcloudimg.com
Transfer of pre-trained representations improves sample efficiency and simplifies
hyperparameter tuning when training deep neural networks for vision. We revisit the …

[PDF][PDF] Supervised representation learning: Transfer learning with deep autoencoders

F Zhuang, X Cheng, P Luo, SJ Pan… - Twenty-fourth international …, 2015 - cse.cuhk.edu.hk
Transfer learning has attracted a lot of attention in the past decade. One crucial research
issue in transfer learning is how to find a good representation for instances of different …

Supervised representation learning with double encoding-layer autoencoder for transfer learning

F Zhuang, X Cheng, P Luo, SJ Pan, Q He - ACM Transactions on …, 2017 - dl.acm.org
Transfer learning has gained a lot of attention and interest in the past decade. One crucial
research issue in transfer learning is how to find a good representation for instances of …

Transtailor: Pruning the pre-trained model for improved transfer learning

B Liu, Y Cai, Y Guo, X Chen - Proceedings of the AAAI conference on …, 2021 - ojs.aaai.org
The increasing of pre-trained models has significantly facilitated the performance on limited
data tasks with transfer learning. However, progress on transfer learning mainly focuses on …

A comprehensive survey on transfer learning

F Zhuang, Z Qi, K Duan, D Xi, Y Zhu… - Proceedings of the …, 2020 - ieeexplore.ieee.org
Transfer learning aims at improving the performance of target learners on target domains by
transferring the knowledge contained in different but related source domains. In this way, the …

Overwriting pretrained bias with finetuning data

A Wang, O Russakovsky - Proceedings of the IEEE/CVF …, 2023 - openaccess.thecvf.com
Transfer learning is beneficial by allowing the expressive features of models pretrained on
large-scale datasets to be finetuned for the target task of smaller, more domain-specific …

Big transfer (bit): General visual representation learning

A Kolesnikov, L Beyer, X Zhai, J Puigcerver… - Computer Vision–ECCV …, 2020 - Springer
Transfer of pre-trained representations improves sample efficiency and simplifies
hyperparameter tuning when training deep neural networks for vision. We revisit the …

Transfer learning via minimizing the performance gap between domains

B Wang, J Mendez, M Cai… - Advances in neural …, 2019 - proceedings.neurips.cc
We propose a new principle for transfer learning, based on a straightforward intuition: if two
domains are similar to each other, the model trained on one domain should also perform …