Aggregated predictors are obtained by making a set of basic predictors vote according to
some weights, that is, to some probability distribution. Randomized predictors are obtained …

Numerous deep learning algorithms have been inspired by and understood via the notion of
information bottleneck, where unnecessary information is (often implicitly) minimized while …

We provide an information-theoretic framework for studying the generalization properties of
machine learning algorithms. Our framework ties together existing approaches, including …

An information-theoretic upper bound on the generalization error of supervised learning
algorithms is derived. The bound is constructed in terms of the mutual information between …

In this work, we improve upon the stepwise analysis of noisy iterative learning algorithms
initiated by Pensia, Jog, and Loh (2018) and recently extended by Bu, Zou, and Veeravalli …

Modern machine learning models are complex and frequently encode surprising amounts of
information about individual inputs. In extreme cases, complex models appear to memorize …

The information-theoretic framework of Russo and Zou (2016) and Xu and Raginsky (2017)
provides bounds on the generalization error of a learning algorithm in terms of the mutual …

Abstract The Predictive Information is the mutual information between the past and the
future, I (Xpast; Xfuture). We hypothesize that capturing the predictive information is useful in …

We derive information-theoretic generalization bounds for supervised learning algorithms
based on the information contained in predictions rather than in the output of the training …

Bounding the generalization error of learning algorithms has a long history, which yet falls
short in explaining various generalization successes including those of deep learning. Two …