Information bottleneck: Exact analysis of (quantized) neural networks
The information bottleneck (IB) principle has been suggested as a way to analyze deep
neural networks. The learning dynamics are studied by inspecting the mutual information
(MI) between the hidden layers and the input and output. Notably, separate fitting and
compression phases during training have been reported. This led to some controversy
including claims that the observations are not reproducible and strongly dependent on the
type of activation function used as well as on the way the MI is estimated. Our study confirms …
neural networks. The learning dynamics are studied by inspecting the mutual information
(MI) between the hidden layers and the input and output. Notably, separate fitting and
compression phases during training have been reported. This led to some controversy
including claims that the observations are not reproducible and strongly dependent on the
type of activation function used as well as on the way the MI is estimated. Our study confirms …
Information Bottleneck: Exact Analysis of (Quantized) Neural Networks
The information bottleneck (IB) principle has been suggested as a way to analyze deep
neural networks. The learning dynamics are studied by inspecting the mutual information
(MI) between the hidden layers and the input and output. Notably, separate fitting and
compression phases during training have been reported. This led to some controversy
including claims that the observations are not reproducible and strongly dependent on the
type of activation function used as well as on the way the MI is estimated. Our study confirms …
neural networks. The learning dynamics are studied by inspecting the mutual information
(MI) between the hidden layers and the input and output. Notably, separate fitting and
compression phases during training have been reported. This led to some controversy
including claims that the observations are not reproducible and strongly dependent on the
type of activation function used as well as on the way the MI is estimated. Our study confirms …
以上显示的是最相近的搜索结果。 查看全部搜索结果