The estimation of mutual information (MI) or conditional mutual information (CMI) from a set
of samples is a longstanding problem. A recent line of work in this area has leveraged the …

Recently, mutual information (MI) has become widely recognized as a statistical measure of
dependence that is suitable for applications where data are non-Gaussian, or where the …

Estimation of information theoretic quantities such as mutual information and its conditional
variant has drawn interest in recent times owing to their multifaceted applications. Newly …

Abstract Information Noise Contrastive Estimation (InfoNCE) is a popular neural estimator of
mutual information (MI). While InfoNCE has demonstrated impressive results in …

Learning representations in an unsupervised or self-supervised manner is a growing area of
research. Current approaches in representation learning seek to maximize the mutual …

Mutual information is a popular metric in machine learning. In case of a discrete target
variable and a continuous feature variable the mutual information can be calculated as a …

We propose a novel approach to the problem of mutual information (MI) estimation via
introducing normalizing flows-based estimator. The estimator maps original data to the …

Abstract Conditional Mutual Information (CMI) is a measure of conditional dependence
between random variables X and Y, given another random variable Z. It can be used to …

Mutual information is a measure of the dependence between random variables that has
been used successfully in myriad applications in many fields. Generalized mutual …

Squared-loss mutual information (SMI) is a robust measure of the statistical dependence
between random variables. The sample-based SMI approximator called least-squares …