Although deep learning has made great progress in recent years, the exploding economic
and environmental costs of training neural networks are becoming unsustainable. To …

Abstract Polyak-Łojasiewicz (PL)(Polyak, 1963) condition is a weaker condition than the
strong convexity but suffices to ensure a global convergence for the Gradient Descent …

This paper studies accelerated gradient methods for nonconvex optimization with Lipschitz
continuous gradient and Hessian. We propose two simple accelerated gradient methods …

We develop an algorithmic framework for solving convex optimization problems using no-
regret game dynamics. By converting the problem of minimizing a convex function into an …

In this work, we show that the heavy-ball ($\HB $) method provably does not reach an
accelerated convergence rate on smooth strongly convex problems. More specifically, we …

We consider a setting that a model needs to adapt to a new domain under distribution shifts,
given that only unlabeled test samples from the new domain are accessible at test time. A …

Invex programs are a special kind of non-convex problems which attain global minima at
every stationary point. While classical first-order gradient descent methods can solve them …

Quasar convexity is a condition that allows some first-order methods to efficiently minimize a
function even when the optimization landscape is non-convex. Previous works develop near …

Stochastic gradient descent with momentum, also known as Stochastic Heavy Ball method
(SHB), is one of the most popular algorithms for solving large-scale stochastic optimization …

Abstract We study\(l_0\)-synthesis/analysis methods and the thresholding-based algorithms
for the dictionary-sparse recovery from a few linear measurements perturbed with Gaussian …