Large Language Models (LLMs) have achieved remarkable success with their billion-level
parameters, yet they incur high inference overheads. The emergence of activation sparsity in …

Large Language Models (LLMs) have dramatically advanced AI applications, yet their
deployment remains challenging due to their immense inference costs. Recent studies …

Large language models (LLMs) have demonstrated proficiency across various natural
language processing (NLP) tasks but often require additional training, such as continual pre …

Exploiting activation sparsity is a promising approach to significantly accelerating the
inference process of large language models (LLMs) without compromising performance …

Large Language Models (LLMs) with billions of parameters have drastically transformed AI
applications. However, their demanding computation during inference has raised significant …

Sparse computation offers a compelling solution for the inference of Large Language
Models (LLMs) in low-resource scenarios by dynamically skipping the computation of …

We introduce, Q-Sparse, a simple yet effective approach to training sparsely-activated large
language models (LLMs). Q-Sparse enables full sparsity of activations in LLMs which can …

Activation sparsity refers to the existence of considerable weakly-contributed elements
among activation outputs. As a prevalent property of the models using the ReLU activation …

Sparse activation, which selectively activates only an input-dependent set of neurons in
inference, is a useful technique to reduce the computing cost of Large Language Models …

Large Language Models (LLMs) have become pivotal in advancing the field of artificial
intelligence, yet their immense sizes pose significant challenges for both fine-tuning and …