User-friendly introduction to PAC-Bayes bounds

P Alquier - Foundations and Trends® in Machine Learning, 2024 - nowpublishers.com
Aggregated predictors are obtained by making a set of basic predictors vote according to
some weights, that is, to some probability distribution. Randomized predictors are obtained …

An optimization-centric view on Bayes' rule: Reviewing and generalizing variational inference

J Knoblauch, J Jewson, T Damoulas - Journal of Machine Learning …, 2022 - jmlr.org
We advocate an optimization-centric view of Bayesian inference. Our inspiration is the
representation of Bayes' rule as infinite-dimensional optimization (Csisz´ r, 1975; Donsker …

A primer on PAC-Bayesian learning

B Guedj - arXiv preprint arXiv:1901.05353, 2019 - arxiv.org
Generalised Bayesian learning algorithms are increasingly popular in machine learning,
due to their PAC generalisation properties and flexibility. The present paper aims at …

A unified recipe for deriving (time-uniform) PAC-Bayes bounds

B Chugg, H Wang, A Ramdas - Journal of Machine Learning Research, 2023 - jmlr.org
We present a unified framework for deriving PAC-Bayesian generalization bounds. Unlike
most previous literature on this topic, our bounds are anytime-valid (ie, time-uniform) …

Meta-learning by adjusting priors based on extended PAC-Bayes theory

R Amit, R Meir - International Conference on Machine …, 2018 - proceedings.mlr.press
In meta-learning an agent extracts knowledge from observed tasks, aiming to facilitate
learning of novel future tasks. Under the assumption that future tasks are 'related'to previous …

An exact characterization of the generalization error for the Gibbs algorithm

G Aminian, Y Bu, L Toni… - Advances in Neural …, 2021 - proceedings.neurips.cc
Various approaches have been developed to upper bound the generalization error of a
supervised learning algorithm. However, existing bounds are often loose and lack of …

A review on transferability estimation in deep transfer learning

Y Xue, R Yang, X Chen, W Liu… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Deep transfer learning has become increasingly prevalent in various fields such as industry
and medical science in recent years. To ensure the successful implementation of target …

On the role of data in PAC-Bayes bounds

GK Dziugaite, K Hsu, W Gharbieh… - International …, 2021 - proceedings.mlr.press
The dominant term in PAC-Bayes bounds is often the Kullback-Leibler divergence between
the posterior and prior. For so-called linear PAC-Bayes risk bounds based on the empirical …

Data-dependent PAC-Bayes priors via differential privacy

GK Dziugaite, DM Roy - Advances in neural information …, 2018 - proceedings.neurips.cc
Abstract The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999)
can incorporate knowledge about the learning algorithm and (data) distribution through the …

Online pac-bayes learning

M Haddouche, B Guedj - Advances in Neural Information …, 2022 - proceedings.neurips.cc
Most PAC-Bayesian bounds hold in the batch learning setting where data is collected at
once, prior to inference or prediction. This somewhat departs from many contemporary …