In this survey we review the concepts and methods of communication systems equipped
with side information. We focus on the channel coding problem, where side information is …

Abstract Information theory provides a range of useful methods to analyse probability
distributions and these techniques have been successfully applied to measure information …

In this paper, we study the vector Gaussian Chief Executive Officer (CEO) problem under
logarithmic loss distortion measure. Specifically, K≥ 2 agents observe independently …

We propose two related iterative algorithms for computing the capacity of discrete
memoryless channels. The celebrated Blahut-Arimoto algorithm is a special case of our …

For general memoryless systems, the existing information-theoretic solutions have a
ldquosingle-letterrdquo form. This reflects the fact that optimum performance can be …

The capacity of a channel can usually be characterized as a maximization of certain entropic
quantities. From a practical point of view it is of primary interest to not only compute the …

Iterative minimization algorithms appear in various areas including machine learning, neural
networks, and information theory. The em algorithm is one of the famous iterative …

In this paper, we extend the Blahut-Arimoto algorithm for maximizing Massey's directed
information. The algorithm can be used for estimating the capacity of channels with delayed …

There has been growing interest in computing rate-distortion functions for real-world data, as
they can provide a theoretical benchmark for compression problems. However, a …

In this paper, we study the vector Gaussian Chief Executive Officer (CEO) problem under
logarithmic loss distortion measure. Specifically, K> 2 agents observe independently …