On information plane analyses of neural network classifiers—A review
BC Geiger - IEEE Transactions on Neural Networks and …, 2021 - ieeexplore.ieee.org
We review the current literature concerned with information plane (IP) analyses of neural
network (NN) classifiers. While the underlying information bottleneck theory and the claim …
network (NN) classifiers. While the underlying information bottleneck theory and the claim …
SNIB: improving spike-based machine learning using nonlinear information bottleneck
S Yang, B Chen - IEEE Transactions on Systems, Man, and …, 2023 - ieeexplore.ieee.org
Spiking neural networks (SNNs) have garnered increased attention in the field of artificial
general intelligence (AGI) research due to their low power consumption, high computational …
general intelligence (AGI) research due to their low power consumption, high computational …
Land cover classification from fused DSM and UAV images using convolutional neural networks
In recent years, remote sensing researchers have investigated the use of different modalities
(or combinations of modalities) for classification tasks. Such modalities can be extracted via …
(or combinations of modalities) for classification tasks. Such modalities can be extracted via …
The information bottleneck problem and its applications in machine learning
Z Goldfeld, Y Polyanskiy - IEEE Journal on Selected Areas in …, 2020 - ieeexplore.ieee.org
Inference capabilities of machine learning (ML) systems skyrocketed in recent years, now
playing a pivotal role in various aspect of society. The goal in statistical learning is to use …
playing a pivotal role in various aspect of society. The goal in statistical learning is to use …
Disco: Remedying self-supervised learning on lightweight models with distilled contrastive learning
Abstract While Self-Supervised Learning (SSL) has received widespread attention from the
community, recent researches argue that its performance often suffers a cliff fall when the …
community, recent researches argue that its performance often suffers a cliff fall when the …
Information flow in deep neural networks
R Shwartz-Ziv - arXiv preprint arXiv:2202.06749, 2022 - arxiv.org
Although deep neural networks have been immensely successful, there is no
comprehensive theoretical understanding of how they work or are structured. As a result …
comprehensive theoretical understanding of how they work or are structured. As a result …
Knowledge consistency between neural networks and beyond
This paper aims to analyze knowledge consistency between pre-trained deep neural
networks. We propose a generic definition for knowledge consistency between neural …
networks. We propose a generic definition for knowledge consistency between neural …
Information plane analysis of deep neural networks via matrix-based Renyi's entropy and tensor kernels
Analyzing deep neural networks (DNNs) via information plane (IP) theory has gained
tremendous attention recently as a tool to gain insight into, among others, their …
tremendous attention recently as a tool to gain insight into, among others, their …
A Survey on Information Bottleneck
This survey is for the remembrance of one of the creators of the information bottleneck
theory, Prof. Naftali Tishby, passing away at the age of 68 on August, 2021. Information …
theory, Prof. Naftali Tishby, passing away at the age of 68 on August, 2021. Information …
Training very deep neural networks: Rethinking the role of skip connections
OK Oyedotun, K Al Ismaeil, D Aouada - Neurocomputing, 2021 - Elsevier
State-of-the-art deep neural networks (DNNs) typically consist of several layers of features
representations, and especially rely on skip connections to avoid the difficulty of model …
representations, and especially rely on skip connections to avoid the difficulty of model …