A review of nonlinear FFT-based computational homogenization methods
M Schneider - Acta Mechanica, 2021 - Springer
Since their inception, computational homogenization methods based on the fast Fourier
transform (FFT) have grown in popularity, establishing themselves as a powerful tool …
transform (FFT) have grown in popularity, establishing themselves as a powerful tool …
Recent advances in stochastic gradient descent in deep learning
In the age of artificial intelligence, the best approach to handling huge amounts of data is a
tremendously motivating and hard problem. Among machine learning models, stochastic …
tremendously motivating and hard problem. Among machine learning models, stochastic …
An improved analysis of stochastic gradient descent with momentum
SGD with momentum (SGDM) has been widely applied in many machine learning tasks, and
it is often applied with dynamic stepsizes and momentum weights tuned in a stagewise …
it is often applied with dynamic stepsizes and momentum weights tuned in a stagewise …
Accelerating federated learning via momentum gradient descent
Federated learning (FL) provides a communication-efficient approach to solve machine
learning problems concerning distributed data, without sending raw data to a central server …
learning problems concerning distributed data, without sending raw data to a central server …
On the linear speedup analysis of communication efficient momentum SGD for distributed non-convex optimization
Recent developments on large-scale distributed machine learning applications, eg, deep
neural networks, benefit enormously from the advances in distributed non-convex …
neural networks, benefit enormously from the advances in distributed non-convex …
On the convergence of a class of adam-type algorithms for non-convex optimization
This paper studies a class of adaptive gradient based momentum algorithms that update the
search directions and learning rates simultaneously using past gradients. This class, which …
search directions and learning rates simultaneously using past gradients. This class, which …
Negative momentum for improved game dynamics
Games generalize the single-objective optimization paradigm by introducing different
objective functions for different players. Differentiable games often proceed by simultaneous …
objective functions for different players. Differentiable games often proceed by simultaneous …
Momentum and stochastic momentum for stochastic gradient, newton, proximal point and subspace descent methods
N Loizou, P Richtárik - Computational Optimization and Applications, 2020 - Springer
In this paper we study several classes of stochastic optimization algorithms enriched with
heavy ball momentum. Among the methods studied are: stochastic gradient descent …
heavy ball momentum. Among the methods studied are: stochastic gradient descent …
Distributed heavy-ball: A generalization and acceleration of first-order methods with gradient tracking
We study distributed optimization to minimize a sum of smooth and strongly-convex
functions. Recent work on this problem uses gradient tracking to achieve linear convergence …
functions. Recent work on this problem uses gradient tracking to achieve linear convergence …
Handbook of convergence theorems for (stochastic) gradient methods
G Garrigos, RM Gower - arXiv preprint arXiv:2301.11235, 2023 - arxiv.org
This is a handbook of simple proofs of the convergence of gradient and stochastic gradient
descent type methods. We consider functions that are Lipschitz, smooth, convex, strongly …
descent type methods. We consider functions that are Lipschitz, smooth, convex, strongly …