Knowledge distillation: A survey

J Gou, B Yu, SJ Maybank, D Tao - International Journal of Computer Vision, 2021 - Springer
In recent years, deep neural networks have been successful in both industry and academia,
especially for computer vision tasks. The great success of deep learning is mainly due to its …

Distilling object detectors via decoupled features

J Guo, K Han, Y Wang, H Wu… - Proceedings of the …, 2021 - openaccess.thecvf.com
Abstract Knowledge distillation is a widely used paradigm for inheriting information from a
complicated teacher network to a compact student network and maintaining the strong …

When object detection meets knowledge distillation: A survey

Z Li, P Xu, X Chang, L Yang, Y Zhang… - … on Pattern Analysis …, 2023 - ieeexplore.ieee.org
Object detection (OD) is a crucial computer vision task that has seen the development of
many algorithms and models over the years. While the performance of current OD models …

CEKD: Cross-modal edge-privileged knowledge distillation for semantic scene understanding using only thermal images

Z Feng, Y Guo, Y Sun - IEEE Robotics and Automation Letters, 2023 - ieeexplore.ieee.org
Semantic scene understanding using thermal images has received great attention due to the
advantage that thermal imaging cameras could see in challenging illumination conditions …

[PDF][PDF] 深度学习中知识蒸馏研究综述

邵仁荣, 刘宇昂, 张伟, 王骏 - 计算机学报, 2022 - 159.226.43.17
摘要在人工智能迅速发展的今天, 深度神经网络广泛应用于各个研究领域并取得了巨大的成功,
但也同样面临着诸多挑战. 首先, 为了解决复杂的问题和提高模型的训练效果 …

Data-free knowledge transfer: A survey

Y Liu, W Zhang, J Wang, J Wang - arXiv preprint arXiv:2112.15278, 2021 - arxiv.org
In the last decade, many deep learning models have been well trained and made a great
success in various fields of machine intelligence, especially for computer vision and natural …

Dynamic contrastive distillation for image-text retrieval

J Rao, L Ding, S Qi, M Fang, Y Liu… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
The recent advancement in vision-and-language pretraining (VLP) has significantly
improved the performance of cross-modal image-text retrieval (ITR) systems. However, the …

A Review of Knowledge Distillation in Object Detection

N Yadikar, K Ubul - IEEE Access, 2023 - ieeexplore.ieee.org
Target detection is a revolutionary advancement in computer vision that provides the ability
to identify specific targets in images for a wide variety of applications, including but not …

Instance, scale, and teacher adaptive knowledge distillation for visual detection in autonomous driving

Q Lan, Q Tian - IEEE Transactions on Intelligent Vehicles, 2022 - ieeexplore.ieee.org
Efficient visual detection is a crucial component in self-driving perception and lays the
foundation for later planning and control stages. Deep-networks-based visual systems …

Generalized knowledge distillation via relationship matching

HJ Ye, S Lu, DC Zhan - IEEE Transactions on Pattern Analysis …, 2022 - ieeexplore.ieee.org
The knowledge of a well-trained deep neural network (aka the “teacher”) is valuable for
learning similar tasks. Knowledge distillation extracts knowledge from the teacher and …