Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
D Oikonomou, N Loizou - arXiv preprint arXiv:2406.04142, 2024 - arxiv.org
Stochastic gradient descent with momentum, also known as Stochastic Heavy Ball method
(SHB), is one of the most popular algorithms for solving large-scale stochastic optimization …
(SHB), is one of the most popular algorithms for solving large-scale stochastic optimization …