Despite their capacity to convey knowledge, most existing knowledge graphs (KGs) are
created for specific domains using low-resource data sources, especially those in non …

Data-free knowledge distillation (DFKD) is a widely-used strategy for Knowledge Distillation
(KD) whose training data is not available. It trains a lightweight student model with the aid of …

We characterize the statistical efficiency of knowledge transfer through $ n $ samples from a
teacher to a probabilistic student classifier with input space $\mathcal S $ over labels …

Abstract Vision Transformers (ViTs) have achieved significant advancement in computer
vision tasks due to their powerful modeling capacity. However, their performance notably …

Convolutional neural networks (CNNs) are known to be vulnerable to adversarial examples
even in black-box scenarios, posing a significant threat to their reliability and security. Most …

Student learners typically engage in an iterative process of actively updating its hypotheses,
like active learning. While this behavior can be advantageous, there is an inherent risk of …

Abstract Does Knowledge Distillation (KD) really work? Conventional wisdom viewed it as a
knowledge transfer procedure where a perfect mimicry of the student to its teacher is …

In the era of large scale pretrained models, Knowledge Distillation (KD) serves an important
role in transferring the wisdom of computationally heavy teacher models to lightweight …

Few-shot knowledge distillation recently emerged as a viable approach to harness the
knowledge of large-scale pre-trained models, using limited data and computational …

Extremely large image generators offer significant transformative potential across diverse
sectors. It allows users to design specific prompts to generate realistic images through some …