Sharpness-aware gradient matching for domain generalization

P Wang, Z Zhang, Z Lei… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
The goal of domain generalization (DG) is to enhance the generalization capability of the
model learned from a source domain to other unseen domains. The recently developed …

Generalized federated learning via sharpness aware minimization

Z Qu, X Li, R Duan, Y Liu, B Tang… - … conference on machine …, 2022 - proceedings.mlr.press
Federated Learning (FL) is a promising framework for performing privacy-preserving,
distributed learning with a set of clients. However, the data distribution among clients often …

Towards efficient and scalable sharpness-aware minimization

Y Liu, S Mai, X Chen, CJ Hsieh… - Proceedings of the IEEE …, 2022 - openaccess.thecvf.com
Abstract Recently, Sharpness-Aware Minimization (SAM), which connects the geometry of
the loss landscape and generalization, has demonstrated a significant performance boost …

Minimizing the accumulated trajectory error to improve dataset distillation

J Du, Y Jiang, VYF Tan, JT Zhou… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
Abstract Model-based deep learning has achieved astounding successes due in part to the
availability of large-scale real-world data. However, processing such massive amounts of …

Sharpness-aware training for free

J Du, D Zhou, J Feng, V Tan… - Advances in Neural …, 2022 - proceedings.neurips.cc
Modern deep neural networks (DNNs) have achieved state-of-the-art performances but are
typically over-parameterized. The over-parameterization may result in undesirably large …

Make sharpness-aware minimization stronger: A sparsified perturbation approach

P Mi, L Shen, T Ren, Y Zhou, X Sun… - Advances in Neural …, 2022 - proceedings.neurips.cc
Deep neural networks often suffer from poor generalization caused by complex and non-
convex loss landscapes. One of the popular solutions is Sharpness-Aware Minimization …

Gradient norm aware minimization seeks first-order flatness and improves generalization

X Zhang, R Xu, H Yu, H Zou… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
Recently, flat minima are proven to be effective for improving generalization and sharpness-
aware minimization (SAM) achieves state-of-the-art performance. Yet the current definition of …

Enhancing fine-tuning based backdoor defense with sharpness-aware minimization

M Zhu, S Wei, L Shen, Y Fan… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
Backdoor defense, which aims to detect or mitigate the effect of malicious triggers introduced
by attackers, is becoming increasingly critical for machine learning security and integrity …

When do flat minima optimizers work?

J Kaddour, L Liu, R Silva… - Advances in Neural …, 2022 - proceedings.neurips.cc
Recently, flat-minima optimizers, which seek to find parameters in low-loss neighborhoods,
have been shown to improve a neural network's generalization performance over stochastic …

A modern look at the relationship between sharpness and generalization

M Andriushchenko, F Croce, M Müller, M Hein… - arXiv preprint arXiv …, 2023 - arxiv.org
Sharpness of minima is a promising quantity that can correlate with generalization in deep
networks and, when optimized during training, can improve generalization. However …