[PDF][PDF] Entropy power inequality for the Rényi entropy.

SG Bobkov, GP Chistyakov - IEEE Trans. Inf. Theory, 2015 - www-users.cse.umn.edu
Entropy power inequality for the Rényi entropy Page 1 1 Entropy power inequality for the Rényi
entropy Sergey G. Bobkov and Gennadiy P. Chistyakov Abstract—The classical entropy power …

On Rényi entropy power inequalities

E Ram, I Sason - IEEE Transactions on Information Theory, 2016 - ieeexplore.ieee.org
This paper gives improved Rényi entropy power inequalities (R-EPIs). Consider a sum S n=
Σ k= 1 n X k of n independent continuous random vectors taking values on ℝ d, and let …

Survival exponential entropies

K Zografos, S Nadarajah - IEEE transactions on information …, 2005 - ieeexplore.ieee.org
The multivariate survival function of a random vector X is used to define a broad class of
entropy measures. Several properties of the proposed class are studied and explicit …

Adaptive blind deconvolution of linear channels using Renyi's entropy with Parzen window estimation

D Erdogmus, KE Hild, JC Principe… - IEEE Transactions …, 2004 - ieeexplore.ieee.org
Blind deconvolution of linear channels is a fundamental signal processing problem that has
immediate extensions to multiple-channel applications. In this paper, we investigate the …

Unpacking information bottlenecks: Unifying information-theoretic objectives in deep learning

A Kirsch, C Lyle, Y Gal - arXiv preprint arXiv:2003.12537, 2020 - arxiv.org
The Information Bottleneck principle offers both a mechanism to explain how deep neural
networks train and generalize, as well as a regularized objective with which to train models …

The minimum support criterion for blind signal extraction: A limiting case of the strengthened young's inequality

S Cruces, I Durán - Independent Component Analysis and Blind Signal …, 2004 - Springer
In this paper, we address the problem of the blind extraction of a subset of “interesting”
independent signals from a linear mixture. We present a novel criterion for the extraction of …

Using information theoretic distance measures for solving the permutation problem of blind source separation of speech signals

E Hoffmann, D Kolossa, BU Köhler… - EURASIP Journal on …, 2012 - Springer
The problem of blind source separation (BSS) of convolved acoustic signals is of great
interest for many classes of applications. Due to the convolutive mixing process, the source …

On the risk of using Rényi's entropy for blind source separation

DT Pham, F Vrins, M Verleysen - IEEE Transactions on Signal …, 2008 - ieeexplore.ieee.org
Recently, some researchers have suggested Renyi's entropy in its general form as a blind
source separation (BSS) objective function. This was motivated by two arguments:(1) …

Blind separation of instantaneous mixtures of dependent sources

M Castella, P Comon - … on Independent Component Analysis and Signal …, 2007 - Springer
This paper deals with the problem of Blind Source Separation. Contrary to the vast majority
of works, we do not assume the statistical independence between the sources and explicitly …

[PDF][PDF] Scalable training with information bottleneck objectives

A Kirsch, C Lyle, Y Gal - … on Uncertainty and Robustness in Deep …, 2020 - gatsby.ucl.ac.uk
Abstract The Information Bottleneck principle offers both a mechanism to explain how deep
neural networks train and generalize, as well as a regularized objective with which to train …