Normalization techniques in training dnns: Methodology, analysis and application

L Huang, J Qin, Y Zhou, F Zhu, L Liu… - IEEE transactions on …, 2023 - ieeexplore.ieee.org
Normalization techniques are essential for accelerating the training and improving the
generalization of deep neural networks (DNNs), and have successfully been used in various …

Deep learning in electron microscopy

JM Ede - Machine Learning: Science and Technology, 2021 - iopscience.iop.org
Deep learning is transforming most areas of science and technology, including electron
microscopy. This review paper offers a practical perspective aimed at developers with …

Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging

S Azizi, L Culp, J Freyberg, B Mustafa, S Baur… - Nature Biomedical …, 2023 - nature.com
Abstract Machine-learning models for medical tasks can match or surpass the performance
of clinical experts. However, in settings differing from those of the training dataset, the …

Analyzing and improving the training dynamics of diffusion models

T Karras, M Aittala, J Lehtinen… - Proceedings of the …, 2024 - openaccess.thecvf.com
Diffusion models currently dominate the field of data-driven image synthesis with their
unparalleled scaling to large datasets. In this paper we identify and rectify several causes for …

Knowledge distillation: A good teacher is patient and consistent

L Beyer, X Zhai, A Royer, L Markeeva… - Proceedings of the …, 2022 - openaccess.thecvf.com
There is a growing discrepancy in computer vision between large-scale models that achieve
state-of-the-art performance and models that are affordable in practical applications. In this …

Big transfer (bit): General visual representation learning

A Kolesnikov, L Beyer, X Zhai, J Puigcerver… - Computer Vision–ECCV …, 2020 - Springer
Transfer of pre-trained representations improves sample efficiency and simplifies
hyperparameter tuning when training deep neural networks for vision. We revisit the …

Deep learning for COVID-19 detection based on CT images

W Zhao, W Jiang, X Qiu - Scientific Reports, 2021 - nature.com
COVID-19 has tremendously impacted patients and medical systems globally. Computed
tomography images can effectively complement the reverse transcription-polymerase chain …

MMDetection: Open mmlab detection toolbox and benchmark

K Chen, J Wang, J Pang, Y Cao, Y Xiong, X Li… - arXiv preprint arXiv …, 2019 - arxiv.org
We present MMDetection, an object detection toolbox that contains a rich set of object
detection and instance segmentation methods as well as related components and modules …

Tinytl: Reduce memory, not parameters for efficient on-device learning

H Cai, C Gan, L Zhu, S Han - Advances in Neural …, 2020 - proceedings.neurips.cc
Efficient on-device learning requires a small memory footprint at training time to fit the tight
memory constraint. Existing work solves this problem by reducing the number of trainable …

Large-scale representation learning on graphs via bootstrapping

S Thakoor, C Tallec, MG Azar, M Azabou… - arXiv preprint arXiv …, 2021 - arxiv.org
Self-supervised learning provides a promising path towards eliminating the need for costly
label information in representation learning on graphs. However, to achieve state-of-the-art …