Unlike existing knowledge distillation methods focus on the baseline settings, where the
teacher models and training strategies are not that strong and competing as state-of-the-art …

The representation gap between teacher and student is an emerging topic in knowledge
distillation (KD). To reduce the gap and improve the performance, current methods often …

Abstract Knowledge distillation involves transferring soft labels from a teacher to a student
using a shared temperature-based softmax function. However the assumption of a shared …

Abstract Knowledge distillation (KD) has witnessed its powerful capability in learning
compact models in object detection. Previous KD methods for object detection mostly focus …

In this paper, we strive to answer the question'how to collaboratively learn convolutional
neural network (CNN)-based and vision transformer (ViT)-based models by selecting and …

In knowledge distillation, previous feature distillation methods mainly focus on the design of
loss functions and the selection of the distilled layers, while the effect of the feature projector …

Abstract Artificial Intelligence of Things (AIoT) which integrates artificial intelligence (AI) and
the Internet of Things (IoT), has attracted increasing attention recently. With the remarkable …

Intelligent video surveillance is an important computer vision application in natural
environments. Since detected objects under surveillance are usually low-resolution and …

With the continual expansion of face datasets, feature-based distillation prevails for large-
scale face recognition. In this work, we attempt to remove identity supervision in student …

Transformer-based architectures have become the de-facto standard models for diverse
vision tasks owing to their superior performance. As the size of these transformer-based …