Fine-tuning visual models has been widely shown promising performance on many
downstream visual tasks. With the surprising development of pre-trained visual foundation …

Influenced by the great success of deep learning via cloud computing and the rapid
development of edge chips, research in artificial intelligence (AI) has shifted to both of the …

The rapid advancement of chat-based language models has led to remarkable progress in
complex task-solving. However, their success heavily relies on human input to guide the …

Abstract Knowledge distillation aims to compress a powerful yet cumbersome teacher model
into a lightweight student model without much sacrifice of performance. For this purpose …

In recent years, deep neural networks have been successful in both industry and academia,
especially for computer vision tasks. The great success of deep learning is mainly due to its …

Abstract Knowledge distillation becomes a de facto standard to improve the performance of
small neural networks. Most of the previous works propose to regress the representational …

Abstract Knowledge Distillation has shown very promising ability in transferring learned
representation from the larger model (teacher) to the smaller one (student). Despite many …

Federated learning is an important privacy-preserving multi-party learning paradigm,
involving collaborative learning with others and local updating on private data. Model …

Pre-training a vison-language model and then fine-tuning it on downstream tasks have
become a popular paradigm. However, pre-trained vison-language models with the …

Abstract Knowledge distillation conditioned on intermediate feature representations always
leads to significant performance improvements. Conventional feature distillation framework …