Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges

B Bischl, M Binder, M Lang, T Pielok… - … : Data Mining and …, 2023 - Wiley Online Library
Most machine learning algorithms are configured by a set of hyperparameters whose values
must be carefully chosen and which often considerably impact performance. To avoid a time …

Recent advances in Bayesian optimization

X Wang, Y Jin, S Schmitt, M Olhofer - ACM Computing Surveys, 2023 - dl.acm.org
Bayesian optimization has emerged at the forefront of expensive black-box optimization due
to its data efficiency. Recent years have witnessed a proliferation of studies on the …

BoTorch: A framework for efficient Monte-Carlo Bayesian optimization

M Balandat, B Karrer, D Jiang… - Advances in neural …, 2020 - proceedings.neurips.cc
Bayesian optimization provides sample-efficient global optimization for a broad range of
applications, including automatic machine learning, engineering, physics, and experimental …

[PDF][PDF] Hyperparameter optimization

M Feurer, F Hutter - Automated machine learning: Methods …, 2019 - library.oapen.org
Recent interest in complex and computationally expensive machine learning models with
many hyperparameters, such as automated machine learning (AutoML) frameworks and …

Learning to optimize: A primer and a benchmark

T Chen, X Chen, W Chen, H Heaton, J Liu… - Journal of Machine …, 2022 - jmlr.org
Learning to optimize (L2O) is an emerging approach that leverages machine learning to
develop optimization methods, aiming at reducing the laborious iterations of hand …

Scalable global optimization via local Bayesian optimization

D Eriksson, M Pearce, J Gardner… - Advances in neural …, 2019 - proceedings.neurips.cc
Bayesian optimization has recently emerged as a popular method for the sample-efficient
optimization of expensive black-box functions. However, the application to high-dimensional …

A tutorial on Bayesian optimization

PI Frazier - arXiv preprint arXiv:1807.02811, 2018 - arxiv.org
Bayesian optimization is an approach to optimizing objective functions that take a long time
(minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of …

Bayesian optimization

PI Frazier - Recent advances in optimization and modeling …, 2018 - pubsonline.informs.org
Bayesian optimization is an approach to optimizing objective functions that take a long time
(minutes or hours) to evaluate. It is best suited for optimization over continuous domains of …

Taking the human out of the loop: A review of Bayesian optimization

B Shahriari, K Swersky, Z Wang… - Proceedings of the …, 2015 - ieeexplore.ieee.org
Big Data applications are typically associated with systems involving large numbers of
users, massive complex software systems, and large-scale heterogeneous computing and …

Show, attend and tell: Neural image caption generation with visual attention

K Xu, J Ba, R Kiros, K Cho, A Courville… - International …, 2015 - proceedings.mlr.press
Inspired by recent work in machine translation and object detection, we introduce an
attention based model that automatically learns to describe the content of images. We …