Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise
Despite the vast empirical evidence supporting the efficacy of adaptive optimization methods
in deep learning, their theoretical understanding is far from complete. This work introduces …
in deep learning, their theoretical understanding is far from complete. This work introduces …
Towards the Generalization and Convergence of Meta Learning Algorithms in Machine Learning
C Stockman - 2024 - rave.ohiolink.edu
Meta learning is an important sub-field of machine learning in which researchers study ways
to train models that can update their parameters well. Many meta-learning papers are …
to train models that can update their parameters well. Many meta-learning papers are …
A comparison of continuous-time approximations to stochastic gradient descent
S Ankirchner, S Perko - Journal of Machine Learning Research, 2024 - jmlr.org
Applying a stochastic gradient descent (SGD) method for minimizing an objective gives rise
to a discrete-time process of estimated parameter values. In order to better understand the …
to a discrete-time process of estimated parameter values. In order to better understand the …