A lot of automatic feedback control and learning tasks carried out on many dynamical
systems still fundamentally rely on a form of proportional–integral–derivative (PID) control …

Lagrangian methods are widely used algorithms for constrained optimization problems, but
their learning dynamics exhibit oscillations and overshoot which, when applied to safe …

Gradient-based optimization algorithms can be studied from the perspective of limiting
ordinary differential equations (ODEs). Motivated by the fact that existing ODEs do not …

In this paper, we develop a unified framework capable of certifying both exponential and
subexponential convergence rates for a wide range of iterative first-order optimization …

Accelerated algorithms have broad applications in large-scale optimization, due to their
generality and fast convergence. However, their stability in the practical setting of noise …

High-performance optimization algorithms are essential in deep learning. However,
understanding the behavior of optimization (ie, learning process) remains challenging due …

Attempts from different disciplines to provide a fundamental understanding of deep learning
have advanced rapidly in recent years, yet a unified framework remains relatively limited. In …

In this paper, we provide a unified analysis of temporal difference learning algorithms with
linear function approximators by exploiting their connections to Markov jump linear systems …

We take a Hamiltonian-based perspective to generalize Nesterov's accelerated gradient
descent and Polyak's heavy ball method to a broad class of momentum methods in the …

Making the gradients small is a fundamental optimization problem that has eluded unifying
and simple convergence arguments in first-order optimization, so far primarily reserved for …