User-friendly introduction to PAC-Bayes bounds

P Alquier - Foundations and Trends® in Machine Learning, 2024 - nowpublishers.com
Aggregated predictors are obtained by making a set of basic predictors vote according to
some weights, that is, to some probability distribution. Randomized predictors are obtained …

Towards learning trustworthily, automatically, and with guarantees on graphs: An overview

L Oneto, N Navarin, B Biggio, F Errica, A Micheli… - Neurocomputing, 2022 - Elsevier
The increasing digitization and datification of all aspects of people's daily life, and the
consequent growth in the use of personal data, are increasingly challenging the current …

Generalization bounds: Perspectives from information theory and PAC-Bayes

F Hellström, G Durisi, B Guedj, M Raginsky - arXiv preprint arXiv …, 2023 - arxiv.org
A fundamental question in theoretical machine learning is generalization. Over the past
decades, the PAC-Bayesian approach has been established as a flexible framework to …

Generalization bounds via convex analysis

G Lugosi, G Neu - Conference on Learning Theory, 2022 - proceedings.mlr.press
Since the celebrated works of Russo and Zou (2016, 2019) and Xu and Raginsky (2017), it
has been well known that the generalization error of supervised learning algorithms can be …

A new family of generalization bounds using samplewise evaluated CMI

F Hellström, G Durisi - Advances in Neural Information …, 2022 - proceedings.neurips.cc
We present a new family of information-theoretic generalization bounds, in which the
training loss and the population loss are compared through a jointly convex function. This …

Sample-conditioned hypothesis stability sharpens information-theoretic generalization bounds

Z Wang, Y Mao - Advances in Neural Information …, 2024 - proceedings.neurips.cc
We present new information-theoretic generalization guarantees through the a novel
construction of the" neighboring-hypothesis" matrix and a new family of stability notions …

Limitations of information-theoretic generalization bounds for gradient descent methods in stochastic convex optimization

M Haghifam, B Rodríguez-Gálvez… - International …, 2023 - proceedings.mlr.press
To date, no “information-theoretic” frameworks for reasoning about generalization error have
been shown to establish minimax rates for gradient descent in the setting of stochastic …

[HTML][HTML] Do we really need a new theory to understand over-parameterization?

L Oneto, S Ridella, D Anguita - Neurocomputing, 2023 - Elsevier
This century saw an unprecedented increase of public and private investments in Artificial
Intelligence (AI) and especially in (Deep) Machine Learning (ML). This led to breakthroughs …

A unified framework for information-theoretic generalization bounds

Y Chu, M Raginsky - Advances in Neural Information …, 2023 - proceedings.neurips.cc
This paper presents a general methodology for deriving information-theoretic generalization
bounds for learning algorithms. The main technical tool is a probabilistic decorrelation …

Explaining generalization power of a dnn using interactive concepts

H Zhou, H Zhang, H Deng, D Liu, W Shen… - Proceedings of the …, 2024 - ojs.aaai.org
This paper explains the generalization power of a deep neural network (DNN) from the
perspective of interactions. Although there is no universally-accepted definition of the …