Early-exit neural networks (EENNs) facilitate adaptive inference by producing predictions at
multiple stages of the forward pass. In safety-critical applications, these predictions are only …

The emerging field of Artificial Intelligence of Things (AIoT) is propelled by the remarkable
success of deep learning and hardware evolution, which has a significant impact on our …

The intriguing in-context learning (ICL) abilities of deep Transformer models have lately
garnered significant attention. By studying in-context linear regression on unimodal …

Scaling machine learning models significantly improves their performance. However, such
gains come at the cost of inference being slow and resource-intensive. Early-exit neural …

Diffusion models have achieved unprecedented performance in image generation, yet they
suffer from slow inference due to their iterative sampling process. To address this, early …

Deploying large language model inference remains challenging due to their high
computational overhead. Early exiting accelerates model inference by adaptively reducing …

Reducing serving cost and latency is a fundamental concern for the deployment of language
models (LMs) in business applications. To address this, cascades of LMs offer an effective …

Ensembling multiple models has always been an effective approach to push the limits of
existing performance and is widely used in classification tasks by simply averaging the …

Increasing the size of large language models (LLMs) has been shown to lead to better
performance. However, this comes at the cost of slower and more expensive inference. Early …