Lightweight Model Pre-Training Via Language Guided Knowledge Distillation

M Li, L Zhang, M Zhu, Z Huang, G Yu… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
This paper studies the problem of pre-training for small models, which is essential for many
mobile devices. Current state-of-the-art methods on this problem transfer the …
相关文章

Lightweight Model Pre-training via Language Guided Knowledge Distillation

M Li, L Zhang, M Zhu, Z Huang, G Yu, J Fan… - arXiv preprint arXiv …, 2024 - arxiv.org
This paper studies the problem of pre-training for small models, which is essential for many
mobile devices. Current state-of-the-art methods on this problem transfer the …