Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …

Abstract Large Language Models (LLMs) have transformed natural language processing
tasks successfully. Yet, their large size and high computational needs pose challenges for …

Large language models (LLMs) targeting different deployment scales and sizes are currently
produced by training each variant from scratch; this is extremely compute-intensive. In this …

Abstract Text-to-image (T2I) generation with Stable Diffusion models (SDMs) involves high
computing demands due to billion-scale parameters. To enhance efficiency, recent studies …

Large language models (LLMs) have revolutionized natural language processing with their
exceptional capabilities. However, deploying LLMs on resource-constrained edge devices …

Despite their nearly universal adoption for large language models, the internal workings of
transformers are not well understood. We aim to better understand the impact of removing or …

The increasing size of language models necessitates a thorough analysis across multiple
dimensions to assess trade-offs among crucial hardware metrics such as latency, energy …

Is it always necessary to compute tokens from shallow to deep layers in Transformers? The
continued success of vanilla Transformers and their variants suggests an undoubted" yes" …

In the era of AIGC, the demand for low-budget or even on-device applications of diffusion
models emerged. In terms of compressing the Stable Diffusion models (SDMs), several …

Large Language Models (LLMs) are not only resource-intensive to train but even more
costly to deploy in production. Therefore, recent work has attempted to prune blocks of LLMs …