Zeroth-order (ZO) optimization is a subset of gradient-free optimization that emerges in many
signal processing and machine learning (ML) applications. It is used for solving optimization …

Fine-tuning language models (LMs) has yielded success on diverse downstream tasks, but
as LMs grow in size, backpropagation requires a prohibitively large amount of memory …

Gradient descent ascent (GDA), the simplest single-loop algorithm for nonconvex minimax
optimization, is widely used in practical applications such as generative adversarial …

Abstract Stochastic Gradient Descent-Ascent (SGDA) is one of the most prominent
algorithms for solving min-max optimization and variational inequalities problems (VIP) …

Federated bilevel optimization (FBO) has shown great potential recently in machine learning
and edge computing due to the emerging nested optimization structure in meta-learning …

In the paper, we propose a class of accelerated zeroth-order and first-order momentum
methods for both nonconvex mini-optimization and minimax-optimization. Specifically, we …

Federated bilevel optimization has attracted increasing attention due to emerging machine
learning and communication applications. The biggest challenge lies in computing the …

We study gradient descent-ascent learning dynamics with timescale separation ($\tau $-
GDA) in unconstrained continuous action zero-sum games where the minimizing player …

In this paper, we consider a class of nonconvex-nonconcave minimax problems, ie, NC-PL
minimax problems, whose objective functions satisfy the Polyak-Łojasiewicz (PL) condition …

In the paper, we propose a class of faster adaptive Gradient Descent Ascent (GDA) methods
for solving the nonconvex-strongly-concave minimax problems by using the unified adaptive …