This survey article delves into the emerging and critical area of symbolic knowledge
distillation in large language models (LLMs). As LLMs such as generative pretrained …

Large Language Models (LLMs) have showcased exceptional capabilities in various
domains, attracting significant interest from both academia and industry. Despite their …

This survey presents an in-depth exploration of knowledge distillation (KD) techniques
within the realm of Large Language Models (LLMs), spotlighting the pivotal role of KD in …

Although language modeling has been trending upwards steadily, models available for low-
resourced languages are limited to large multilingual models such as mBERT and XLM …

Large language models (LLMs), as a particular instance of generative artificial intelligence,
have revolutionized natural language processing. In this invited paper, we argue that LLMs …

In the past few years, transformer-based pre-trained language models have achieved
astounding success in both industry and academia. However, the large model size and high …

Large Language Models (LLMs) are versatile, yet they often falter in tasks requiring deep
and reliable reasoning due to issues like hallucinations, limiting their applicability in critical …

Abstract Large Language Models (LLMs) are characterized by their inherent memory
inefficiency and compute-intensive nature, making them impractical to run on low-resource …

The remarkable capabilities of modern large language models are rooted in their vast
repositories of knowledge encoded within their parameters, enabling them to perceive the …

The ability to summarize and organize knowledge into abstract concepts is key to learning
and reasoning. Many industrial applications rely on the consistent and systematic use of …