User-friendly introduction to PAC-Bayes bounds

P Alquier - Foundations and Trends® in Machine Learning, 2024 - nowpublishers.com
Aggregated predictors are obtained by making a set of basic predictors vote according to
some weights, that is, to some probability distribution. Randomized predictors are obtained …

Generalization bounds: Perspectives from information theory and PAC-Bayes

F Hellström, G Durisi, B Guedj… - … and Trends® in …, 2025 - nowpublishers.com
A fundamental question in theoretical machine learning is generalization. Over the past
decades, the PAC-Bayesian approach has been established as a flexible framework to …

PAC-Bayes generalisation bounds for heavy-tailed losses through supermartingales

M Haddouche, B Guedj - arXiv preprint arXiv:2210.00928, 2022 - arxiv.org
While PAC-Bayes is now an established learning framework for light-tailed losses (\emph
{eg}, subgaussian or subexponential), its extension to the case of heavy-tailed losses …

MMD-FUSE: Learning and combining kernels for two-sample testing without data splitting

F Biggs, A Schrab, A Gretton - Advances in Neural …, 2024 - proceedings.neurips.cc
We propose novel statistics which maximise the power of a two-sample test based on the
Maximum Mean Discrepancy (MMD), byadapting over the set of kernels used in defining it …

Learning via Wasserstein-based high probability generalisation bounds

P Viallard, M Haddouche… - Advances in Neural …, 2024 - proceedings.neurips.cc
Minimising upper bounds on the population risk or the generalisation gap has been widely
used in structural risk minimisation (SRM)--this is in particular at the core of PAC-Bayesian …

Non-vacuous generalisation bounds for shallow neural networks

F Biggs, B Guedj - International Conference on Machine …, 2022 - proceedings.mlr.press
We focus on a specific class of shallow neural networks with a single hidden layer, namely
those with $ L_2 $-normalised data and either a sigmoid-shaped Gaussian error function …

On PAC-Bayesian reconstruction guarantees for VAEs

BE Chérief-Abdellatif, Y Shi… - International …, 2022 - proceedings.mlr.press
Despite its wide use and empirical successes, the theoretical understanding and study of the
behaviour and performance of the variational autoencoder (VAE) have only emerged in the …

PAC-Bayesian generalization bounds for adversarial generative models

SD Mbacke, F Clerc, P Germain - … Conference on Machine …, 2023 - proceedings.mlr.press
We extend PAC-Bayesian theory to generative models and develop generalization bounds
for models based on the Wasserstein distance and the total variation distance. Our first result …

On margins and generalisation for voting classifiers

F Biggs, V Zantedeschi, B Guedj - Advances in Neural …, 2022 - proceedings.neurips.cc
We study the generalisation properties of majority voting on finite ensembles of classifiers,
proving margin-based generalisation bounds via the PAC-Bayes theory. These provide state …

Data-dependent generalization bounds via variable-size compressibility

M Sefidgaran, A Zaidi - IEEE Transactions on Information …, 2024 - ieeexplore.ieee.org
In this paper, we establish novel data-dependent upper bounds on the generalization error
through the lens of a “variable-size compressibility” framework that we introduce newly here …