Federated minimax optimization: Improved convergence analyses and algorithms

P Sharma, R Panda, G Joshi… - … on Machine Learning, 2022 - proceedings.mlr.press
In this paper, we consider nonconvex minimax optimization, which is gaining prominence in
many modern machine learning applications, such as GANs. Large-scale edge-based …

Practical and matching gradient variance bounds for black-box variational Bayesian inference

K Kim, K Wu, J Oh, JR Gardner - … Conference on Machine …, 2023 - proceedings.mlr.press
Understanding the gradient variance of black-box variational inference (BBVI) is a crucial
step for establishing its convergence and developing algorithmic improvements. However …

Aiming towards the minimizers: fast convergence of SGD for overparametrized problems

C Liu, D Drusvyatskiy, M Belkin… - Advances in neural …, 2024 - proceedings.neurips.cc
Modern machine learning paradigms, such as deep learning, occur in or close to the
interpolation regime, wherein the number of model parameters is much larger than the …

Accelerated stochastic optimization methods under quasar-convexity

Q Fu, D Xu, AC Wilson - International Conference on …, 2023 - proceedings.mlr.press
Non-convex optimization plays a key role in a growing number of machine learning
applications. This motivates the identification of specialized structure that enables sharper …

Continuized acceleration for quasar convex functions in non-convex optimization

JK Wang, A Wibisono - arXiv preprint arXiv:2302.07851, 2023 - arxiv.org
Quasar convexity is a condition that allows some first-order methods to efficiently minimize a
function even when the optimization landscape is non-convex. Previous works develop near …

On the convergence to a global solution of shuffling-type gradient algorithms

L Nguyen, TH Tran - Advances in Neural Information …, 2024 - proceedings.neurips.cc
Stochastic gradient descent (SGD) algorithm is the method of choice in many machine
learning tasks thanks to its scalability and efficiency in dealing with large-scale problems. In …

Tackling benign nonconvexity with smoothing and stochastic gradients

H Vardhan, SU Stich - arXiv preprint arXiv:2202.09052, 2022 - arxiv.org
Non-convex optimization problems are ubiquitous in machine learning, especially in Deep
Learning. While such complex problems can often be successfully optimized in practice by …