Differential equations have proven to be a powerful mathematical tool in science and
engineering, leading to better understanding, prediction, and control of dynamic processes …

We analyze the convergence rate of various momentum-based optimization algorithms from
a dynamical systems point of view. Our analysis exploits fundamental topological properties …

Recent work has attempted to interpret residual networks (ResNets) as one step of a forward
Euler discretization of an ordinary differential equation, focusing mainly on syntactic …

Fractional calculus is an efficient tool, which has the potential to improve the performance of
gradient methods. However, when the first order gradient direction is generalized by …

We analyze continuous-time models of accelerated gradient methods through deriving
conservation laws in dilated coordinate systems. Namely, instead of analyzing the dynamics …

We revisit the Ravine method of Gelfand and Tsetlin from a dynamical system perspective,
study its convergence properties, and highlight its similarities and differences with the …

We take a Hamiltonian-based perspective to generalize Nesterov's accelerated gradient
descent and Polyak's heavy ball method to a broad class of momentum methods in the …

In this manuscript we study the properties of a family of a second-order differential equations
with damping, its discretizations, and their connections with accelerated optimization …

Recently, continuous-time dynamical systems have proved useful in providing conceptual
and quantitative insights into gradient-based optimization, widely used in modern machine …

The augmented Lagrangian method (ALM) is a classical optimization tool that solves a given
“difficult”(constrained) problem via finding solutions of a sequence of “easier”(often …