Abstract Large Language Models (LLMs) have transformed natural language processing
tasks successfully. Yet, their large size and high computational needs pose challenges for …

The field of efficient Large Language Model (LLM) inference is rapidly evolving, presenting a
unique blend of opportunities and challenges. Although the field has expanded and is …

Large language models (LLMs) have achieved remarkable advancements in natural
language processing, showcasing exceptional performance across various tasks. However …

Post-training quantization (PTQ) serves as a potent technique to accelerate the inference of
large language models (LLMs). Nonetheless, existing works still necessitate a considerable …

Advancements in Large Language Models (LLMs) inspire various strategies for integrating
video modalities. A key approach is Video-LLMs, which incorporate an optimizable interface …

With large language models (LLMs) widely deployed in long content generation recently,
there has emerged an increasing demand for efficient long-sequence inference support …

Small Language Models (SLMs) have become increasingly important due to their efficiency
and performance to perform various language tasks with minimal computational resources …

Transformer-based long context generative models power emerging AI applications like
hour-long video understanding and project-level coding agent. Deploying long context …

With the widespread deployment of long-context large language models (LLMs), there has
been a growing demand for efficient support of high-throughput inference. However, as the …

This paper provides a comprehensive overview of the principles, challenges, and
methodologies associated with quantizing large-scale neural network models. As neural …