Learning representations in an unsupervised or self-supervised manner is a growing area of
research. Current approaches in representation learning seek to maximize the mutual …

Estimating mutual information (MI) from samples is a fundamental problem in statistics,
machine learning, and data analysis. Recently it was shown that a popular class of non …

Squared-loss mutual information (SMI) is a robust measure of the statistical dependence
between random variables. The sample-based SMI approximator called least-squares …

Mutual information (MI) quantifies the statistical dependency between a pair of random
variables, and plays a central role in the analysis of engineering and biological systems …

Abstract Information Noise Contrastive Estimation (InfoNCE) is a popular neural estimator of
mutual information (MI). While InfoNCE has demonstrated impressive results in …

The estimation of mutual information (MI) or conditional mutual information (CMI) from a set
of samples is a longstanding problem. A recent line of work in this area has leveraged the …

Total correlation (TC) is a fundamental concept in information theory that measures
statistical dependency among multiple random variables. Recently, TC has shown …

Quantifying the dependence between two random variables is a fundamental issue in data
analysis, and thus many measures have been proposed. Recent studies have focused on …

We demonstrate that a popular class of non-parametric mutual information (MI) estimators
based on k-nearest-neighbor graphs requires number of samples that scales exponentially …

Mutual information (MI) is a fundamental quantity in information theory and machine
learning. However, direct estimation of MI is intractable, even if the true joint probability …