Large-scale machine learning with fast and stable stochastic conjugate gradient
Z Yang - Computers & Industrial Engineering, 2022 - Elsevier
In deterministic optimization, conjugate gradient (CG) type approaches are preferred with a
superior convergence rate than the ordinary gradient approaches. The requirement of …
superior convergence rate than the ordinary gradient approaches. The requirement of …
Adaptive proximal SGD based on new estimating sequences for sparser ERM
Z Zhang, S Zhou - Information Sciences, 2023 - Elsevier
Estimating sequences introduced by Nesterov is an efficient trick to accelerate gradient
descent (GD). The stochastic version of estimating sequences is also successfully used to …
descent (GD). The stochastic version of estimating sequences is also successfully used to …
SARAH-M: A fast stochastic recursive gradient descent algorithm via momentum
Z Yang - Expert Systems with Applications, 2024 - Elsevier
As a simple but effective way, the momentum method has been widely adopted in stochastic
optimization algorithms for large-scale machine learning problems and the success of …
optimization algorithms for large-scale machine learning problems and the success of …
Accelerating Recursive and Variance Reduced Stochastic Gradient Methods by Adaptive Barzilai-Borwein Step Sizes
J Wang, Y Yang, Z Peng - arXiv preprint arXiv:2307.13930, 2023 - arxiv.org
Mini-Batch version of StochAstic Recursive grAdient algoritHm and Stochastic Variance
Reduced Gradient method, employed random Barzilai-Borwein step size (shorted as MB …
Reduced Gradient method, employed random Barzilai-Borwein step size (shorted as MB …
Painless Stochastic Conjugate Gradient for Large-Scale Machine Learning
Z Yang - IEEE Transactions on Neural Networks and Learning …, 2023 - ieeexplore.ieee.org
Conjugate gradient (CG), as an effective technique to speed up gradient descent algorithms,
has shown great potential and has widely been used for large-scale machine-learning …
has shown great potential and has widely been used for large-scale machine-learning …
General inertial proximal stochastic variance reduction gradient for nonconvex nonsmooth optimization
S Sun, L He - Journal of Inequalities and Applications, 2023 - Springer
In this paper, motivated by the competitive performance of the proximal stochastic variance
reduction gradient (Prox-SVRG) method, a novel general inertial Prox-SVRG (GIProx-SVRG) …
reduction gradient (Prox-SVRG) method, a novel general inertial Prox-SVRG (GIProx-SVRG) …
Variance reduced optimization with implicit gradient transport
Z Yang - Knowledge-Based Systems, 2021 - Elsevier
The question of how to readily reduce variance in stochastic optimization methods is
challenging. The most widely used approach of reducing variance in stochastic optimization …
challenging. The most widely used approach of reducing variance in stochastic optimization …