Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges

B Bischl, M Binder, M Lang, T Pielok… - … : Data Mining and …, 2023 - Wiley Online Library
Most machine learning algorithms are configured by a set of hyperparameters whose values
must be carefully chosen and which often considerably impact performance. To avoid a time …

A systematic review on overfitting control in shallow and deep neural networks

MM Bejani, M Ghatee - Artificial Intelligence Review, 2021 - Springer
Shallow neural networks process the features directly, while deep networks extract features
automatically along with the training. Both models suffer from overfitting or poor …

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

M Wortsman, G Ilharco, SY Gadre… - International …, 2022 - proceedings.mlr.press
The conventional recipe for maximizing model accuracy is to (1) train multiple models with
various hyperparameters and (2) pick the individual model which performs best on a held …

Laplace redux-effortless bayesian deep learning

E Daxberger, A Kristiadi, A Immer… - Advances in …, 2021 - proceedings.neurips.cc
Bayesian formulations of deep learning have been shown to have compelling theoretical
properties and offer practical functional benefits, such as improved predictive uncertainty …

Advances and open problems in federated learning

P Kairouz, HB McMahan, B Avent… - … and trends® in …, 2021 - nowpublishers.com
Federated learning (FL) is a machine learning setting where many clients (eg, mobile
devices or whole organizations) collaboratively train a model under the orchestration of a …

Hyper-parameter optimization: A review of algorithms and applications

T Yu, H Zhu - arXiv preprint arXiv:2003.05689, 2020 - arxiv.org
Since deep neural networks were developed, they have made huge contributions to
everyday lives. Machine learning provides more rational advice than humans are capable of …

Physics-informed neural networks for high-speed flows

Z Mao, AD Jagtap, GE Karniadakis - Computer Methods in Applied …, 2020 - Elsevier
In this work we investigate the possibility of using physics-informed neural networks (PINNs)
to approximate the Euler equations that model high-speed aerodynamic flows. In particular …

[HTML][HTML] Automated machine learning: Review of the state-of-the-art and opportunities for healthcare

J Waring, C Lindvall, R Umeton - Artificial intelligence in medicine, 2020 - Elsevier
Objective This work aims to provide a review of the existing literature in the field of
automated machine learning (AutoML) to help healthcare professionals better utilize …

Recent advances in Bayesian optimization

X Wang, Y Jin, S Schmitt, M Olhofer - ACM Computing Surveys, 2023 - dl.acm.org
Bayesian optimization has emerged at the forefront of expensive black-box optimization due
to its data efficiency. Recent years have witnessed a proliferation of studies on the …

AutoML: A survey of the state-of-the-art

X He, K Zhao, X Chu - Knowledge-based systems, 2021 - Elsevier
Deep learning (DL) techniques have obtained remarkable achievements on various tasks,
such as image recognition, object detection, and language modeling. However, building a …