Make landscape flatter in differentially private federated learning

Y Shi, Y Liu, K Wei, L Shen… - Proceedings of the …, 2023 - openaccess.thecvf.com
To defend the inference attacks and mitigate the sensitive information leakages in Federated
Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy …

Gradient norm aware minimization seeks first-order flatness and improves generalization

X Zhang, R Xu, H Yu, H Zou… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
Recently, flat minima are proven to be effective for improving generalization and sharpness-
aware minimization (SAM) achieves state-of-the-art performance. Yet the current definition of …

Improving the model consistency of decentralized federated learning

Y Shi, L Shen, K Wei, Y Sun, B Yuan… - International …, 2023 - proceedings.mlr.press
To mitigate the privacy leakages and communication burdens of Federated Learning (FL),
decentralized FL (DFL) discards the central server and each client only communicates with …

Enhancing fine-tuning based backdoor defense with sharpness-aware minimization

M Zhu, S Wei, L Shen, Y Fan… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
Backdoor defense, which aims to detect or mitigate the effect of malicious triggers introduced
by attackers, is becoming increasingly critical for machine learning security and integrity …

Robust generalization against photon-limited corruptions via worst-case sharpness minimization

Z Huang, M Zhu, X Xia, L Shen, J Yu… - Proceedings of the …, 2023 - openaccess.thecvf.com
Robust generalization aims to tackle the most challenging data distributions which are rare
in the training set and contain severe noises, ie, photon-limited corruptions. Common …

Flatness-aware minimization for domain generalization

X Zhang, R Xu, H Yu, Y Dong… - Proceedings of the …, 2023 - openaccess.thecvf.com
Abstract Domain generalization (DG) seeks to learn robust models that generalize well
under unknown distribution shifts. As a critical aspect of DG, optimizer selection has not …

Enhancing sharpness-aware optimization through variance suppression

B Li, G Giannakis - Advances in Neural Information …, 2024 - proceedings.neurips.cc
Sharpness-aware minimization (SAM) has well documented merits in enhancing
generalization of deep neural networks, even without sizable data augmentation. Embracing …

Improving sharpness-aware minimization with fisher mask for better generalization on language models

Q Zhong, L Ding, L Shen, P Mi, J Liu, B Du… - arXiv preprint arXiv …, 2022 - arxiv.org
Fine-tuning large pretrained language models on a limited training corpus usually suffers
from poor generalization. Prior works show that the recently-proposed sharpness-aware …

Data augmented flatness-aware gradient projection for continual learning

E Yang, L Shen, Z Wang, S Liu… - Proceedings of the …, 2023 - openaccess.thecvf.com
The goal of continual learning (CL) is to continuously learn new tasks without forgetting
previously learned old tasks. To alleviate catastrophic forgetting, gradient projection based …

Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape

Y Sun, L Shen, S Chen, L Ding… - … Conference on Machine …, 2023 - proceedings.mlr.press
In federated learning (FL), a cluster of local clients are chaired under the coordination of the
global server and cooperatively train one model with privacy protection. Due to the multiple …