Adaptive hierarchical hyper-gradient descent

R Jie, J Gao, A Vasnev, MN Tran - International Journal of Machine …, 2022 - Springer
Adaptive learning rate strategies can lead to faster convergence and better performance for
deep learning models. There are some widely known human-designed adaptive optimizers …

[PDF][PDF] Adaptive Multi-level Hyper-gradient Descent

R Jie, J Gao, A Vasnev, MN Tran - arXiv preprint arXiv:2008.07277, 2020 - academia.edu
In this study, we investigate learning rate adaption at different levels based on the hyper-
gradient descent framework and propose a method that adaptively learns the optimizer …

Advances in Meta-Learning, Robustness, and Second-Order Optimisation in Deep Learning

E Oldewage - 2024 - repository.cam.ac.uk
In machine learning, we are concerned with developing algorithms that are able to learn,
that is, to accumulate knowledge about how to do a task without having been programmed …