Near-optimal algorithms for minimax optimization

T Lin, C Jin, MI Jordan - Conference on Learning Theory, 2020 - proceedings.mlr.press
This paper resolves a longstanding open question pertaining to the design of near-optimal
first-order algorithms for smooth and strongly-convex-strongly-concave minimax problems …

The complexity of constrained min-max optimization

C Daskalakis, S Skoulakis, M Zampetakis - Proceedings of the 53rd …, 2021 - dl.acm.org
Despite its important applications in Machine Learning, min-max optimization of objective
functions that are nonconvex-nonconcave remains elusive. Not only are there no known first …

Efficient algorithms for smooth minimax optimization

KK Thekumparampil, P Jain… - Advances in Neural …, 2019 - proceedings.neurips.cc
This paper studies first order methods for solving smooth minimax optimization problems
$\min_x\max_y g (x, y) $ where $ g (\cdot,\cdot) $ is smooth and $ g (x,\cdot) $ is concave for …

High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance

A Sadiev, M Danilova, E Gorbunov… - International …, 2023 - proceedings.mlr.press
During the recent years the interest of optimization and machine learning communities in
high-probability convergence of stochastic optimization methods has been growing. One of …

Accelerated Algorithms for Smooth Convex-Concave Minimax Problems with O (1/k^ 2) Rate on Squared Gradient Norm

TH Yoon, EK Ryu - International Conference on Machine …, 2021 - proceedings.mlr.press
In this work, we study the computational complexity of reducing the squared gradient
magnitude for smooth minimax optimization problems. First, we present algorithms with …

Fast extra gradient methods for smooth structured nonconvex-nonconcave minimax problems

S Lee, D Kim - Advances in Neural Information Processing …, 2021 - proceedings.neurips.cc
Modern minimax problems, such as generative adversarial network and adversarial training,
are often under a nonconvex-nonconcave setting, and developing an efficient method for …

Last-iterate convergence: Zero-sum games and constrained min-max optimization

C Daskalakis, I Panageas - arXiv preprint arXiv:1807.04252, 2018 - arxiv.org
Motivated by applications in Game Theory, Optimization, and Generative Adversarial
Networks, recent work of Daskalakis et al\cite {DISZ17} and follow-up work of Liang and …

Tight last-iterate convergence rates for no-regret learning in multi-player games

N Golowich, S Pattathil… - Advances in neural …, 2020 - proceedings.neurips.cc
We study the question of obtaining last-iterate convergence rates for no-regret learning
algorithms in multi-player games. We show that the optimistic gradient (OG) algorithm with a …

Federated minimax optimization: Improved convergence analyses and algorithms

P Sharma, R Panda, G Joshi… - … on Machine Learning, 2022 - proceedings.mlr.press
In this paper, we consider nonconvex minimax optimization, which is gaining prominence in
many modern machine learning applications, such as GANs. Large-scale edge-based …

On lower iteration complexity bounds for the convex concave saddle point problems

J Zhang, M Hong, S Zhang - Mathematical Programming, 2022 - Springer
In this paper, we study the lower iteration complexity bounds for finding the saddle point of a
strongly convex and strongly concave saddle point problem: min x max y F (x, y). We restrict …