Gradient-based hyperparameter optimization over long horizons

P Micaelli, AJ Storkey - Advances in Neural Information …, 2021 - proceedings.neurips.cc
Gradient-based hyperparameter optimization has earned a widespread popularity in the
context of few-shot meta-learning, but remains broadly impractical for tasks with long …

Non-greedy gradient-based hyperparameter optimization over long horizons

P Micaelli, A Storkey - 2020 - openreview.net
Gradient-based meta-learning has earned a widespread popularity in few-shot learning, but
remains broadly impractical for tasks with long horizons (many gradient steps), due to …

Meta-learning with implicit gradients

A Rajeswaran, C Finn, SM Kakade… - Advances in neural …, 2019 - proceedings.neurips.cc
A core capability of intelligent systems is the ability to quickly learn new tasks by drawing on
prior experience. Gradient (or optimization) based meta-learning has recently emerged as …

EMO: episodic memory optimization for few-shot meta-learning

Y Du, J Shen, X Zhen… - Conference on Lifelong …, 2023 - proceedings.mlr.press
Few-shot meta-learning presents a challenge for gradient descent optimization due to the
limited number of training samples per task. To address this issue, we propose an episodic …

Improving Generalization in Meta-Learning via Meta-Gradient Augmentation

R Wang, H Sun, Q Wei, X Nie, Y Ma, Y Yin - arXiv preprint arXiv …, 2023 - arxiv.org
Meta-learning methods typically follow a two-loop framework, where each loop potentially
suffers from notorious overfitting, hindering rapid adaptation and generalization to new …

[PDF][PDF] A structured prediction approach for conditional meta-learning

R Wang, Y Demiris, C Ciliberto - Advances in Neural Information …, 2020 - researchgate.net
Optimization-based meta-learning algorithms are a powerful class of methods for learning-to-
learn applications such as few-shot learning. They tackle the limited availability of training …

SHOT: suppressing the hessian along the optimization trajectory for gradient-based meta-learning

JH Lee, J Yoo, N Kwak - Advances in Neural Information …, 2023 - proceedings.neurips.cc
In this paper, we hypothesize that gradient-based meta-learning (GBML) implicitly
suppresses the Hessian along the optimization trajectory in the inner loop. Based on this …

Sharp-maml: Sharpness-aware model-agnostic meta learning

M Abbas, Q Xiao, L Chen, PY Chen… - … on machine learning, 2022 - proceedings.mlr.press
Abstract Model-agnostic meta learning (MAML) is currently one of the dominating
approaches for few-shot meta-learning. Albeit its effectiveness, the optimization of MAML …

Large-scale meta-learning with continual trajectory shifting

J Shin, HB Lee, B Gong, SJ Hwang - arXiv preprint arXiv:2102.07215, 2021 - arxiv.org
Meta-learning of shared initialization parameters has shown to be highly effective in solving
few-shot learning tasks. However, extending the framework to many-shot scenarios, which …

Meta-AdaM: An meta-learned adaptive optimizer with momentum for few-shot learning

S Sun, H Gao - Advances in Neural Information Processing …, 2024 - proceedings.neurips.cc
Abstract We introduce Meta-AdaM, a meta-learned adaptive optimizer with momentum,
designed for few-shot learning tasks that pose significant challenges to deep learning …