In recent years, deep neural networks have been successful in both industry and academia,
especially for computer vision tasks. The great success of deep learning is mainly due to its …

Abstract Graph Neural Networks (GNNs) have shown exceptional performance in the task of
link prediction. Despite their effectiveness, the high latency brought by non-trivial …

Abstract Knowledge Distillation (KD) aims at transferring the knowledge of a well-performed
neural network (the {\it teacher}) to a weaker one (the {\it student}). A peculiar phenomenon …

摘要在人工智能迅速发展的今天, 深度神经网络广泛应用于各个研究领域并取得了巨大的成功,
但也同样面临着诸多挑战. 首先, 为了解决复杂的问题和提高模型的训练效果 …

Abstract Knowledge distillation (KD) can effectively compress neural networks by training a
smaller network (student) to simulate the behavior of a larger one (teacher). A counter …

Abstract Knowledge distillation transfers knowledge from a large model to a small one via
task and distillation losses. In this paper, we observe a trade-off between task and distillation …

Weakly supervised semantic segmentation (WSSS) is a challenging yet important research
field in vision community. In WSSS, the key problem is to generate high-quality pseudo …

Knowledge distillation is a popular technique to transfer knowledge from large teacher
models to a small student model. Typically, the student learns to imitate the teacher by …

Knowledge distillation is widely used as a means of improving the performance of a
relatively simple student model using the predictions from a complex teacher model. Several …

Since the advent of knowledge distillation, many researchers have been intrigued by the
$\textit {dark knowledge} $ hidden in the soft labels generated by the teacher model. This …