To explain how the brain orchestrates information-processing for cognition, we must
understand information itself. Importantly, information is not a monolithic entity. Information …

The goal of this paper is to promote the idea that including semantic and goal-oriented
aspects in future 6G networks can produce a significant leap forward in terms of system …

Reinforcement Learning (RL) is a popular machine learning paradigm where intelligent
agents interact with the environment to fulfill a long-term goal. Driven by the resurgence of …

Abstract While Multiple Instance Learning (MIL) has shown promising results in digital
Pathology Whole Slide Image (WSI) analysis, such a paradigm still faces performance and …

Abstract The Information Bottleneck (IB) provides an information theoretic principle for
representation learning, by retaining all information relevant for predicting label while …

Numerous deep learning algorithms have been inspired by and understood via the notion of
information bottleneck, where unnecessary information is (often implicitly) minimized while …

In this paper, we conduct an empirical study of the feature learning process in deep
classifiers. Recent research has identified a training phenomenon called Neural Collapse …

This paper presents a method to interpret the success of knowledge distillation by
quantifying and analyzing task-relevant and task-irrelevant visual concepts that are encoded …

Previous work has proposed many new loss functions and regularizers that improve test
accuracy on image classification tasks. However, it is not clear whether these loss functions …

Abstract Knowledge distillation is an effective approach to learn compact models (students)
with the supervision of large and strong models (teachers). As empirically there exists a …