Federated minimax optimization: Improved convergence analyses and algorithms
In this paper, we consider nonconvex minimax optimization, which is gaining prominence in
many modern machine learning applications, such as GANs. Large-scale edge-based …
many modern machine learning applications, such as GANs. Large-scale edge-based …
Practical and matching gradient variance bounds for black-box variational Bayesian inference
Understanding the gradient variance of black-box variational inference (BBVI) is a crucial
step for establishing its convergence and developing algorithmic improvements. However …
step for establishing its convergence and developing algorithmic improvements. However …
Aiming towards the minimizers: fast convergence of SGD for overparametrized problems
Modern machine learning paradigms, such as deep learning, occur in or close to the
interpolation regime, wherein the number of model parameters is much larger than the …
interpolation regime, wherein the number of model parameters is much larger than the …
Accelerated stochastic optimization methods under quasar-convexity
Non-convex optimization plays a key role in a growing number of machine learning
applications. This motivates the identification of specialized structure that enables sharper …
applications. This motivates the identification of specialized structure that enables sharper …
Continuized acceleration for quasar convex functions in non-convex optimization
JK Wang, A Wibisono - arXiv preprint arXiv:2302.07851, 2023 - arxiv.org
Quasar convexity is a condition that allows some first-order methods to efficiently minimize a
function even when the optimization landscape is non-convex. Previous works develop near …
function even when the optimization landscape is non-convex. Previous works develop near …
On the convergence to a global solution of shuffling-type gradient algorithms
Stochastic gradient descent (SGD) algorithm is the method of choice in many machine
learning tasks thanks to its scalability and efficiency in dealing with large-scale problems. In …
learning tasks thanks to its scalability and efficiency in dealing with large-scale problems. In …
Tackling benign nonconvexity with smoothing and stochastic gradients
Non-convex optimization problems are ubiquitous in machine learning, especially in Deep
Learning. While such complex problems can often be successfully optimized in practice by …
Learning. While such complex problems can often be successfully optimized in practice by …