Deep Learning Workload Scheduling in GPU Datacenters: A Survey
Deep learning (DL) has demonstrated its remarkable success in a wide variety of fields. The
development of a DL model is a time-consuming and resource-intensive procedure. Hence …
development of a DL model is a time-consuming and resource-intensive procedure. Hence …
Deferred continuous batching in resource-efficient large language model serving
Despite that prior work of batched inference and parameter-efficient fine-tuning techniques
have reduced the resource requirements of large language models (LLMs), challenges …
have reduced the resource requirements of large language models (LLMs), challenges …
LoongServe: Efficiently Serving Long-context Large Language Models with Elastic Sequence Parallelism
The context window of large language models (LLMs) is rapidly increasing, leading to a
huge variance in resource usage between different requests as well as between different …
huge variance in resource usage between different requests as well as between different …
Parrot: Efficient Serving of LLM-based Applications with Semantic Variable
The rise of large language models (LLMs) has enabled LLM-based applications (aka AI
agents or co-pilots), a new software paradigm that combines the strength of LLM and …
agents or co-pilots), a new software paradigm that combines the strength of LLM and …
vTrain: A Simulation Framework for Evaluating Cost-effective and Compute-optimal Large Language Model Training
As large language models (LLMs) become widespread in various application domains, a
critical challenge the AI community is facing is how to train these large AI models in a cost …
critical challenge the AI community is facing is how to train these large AI models in a cost …
MLTCP: Congestion Control for DNN Training
S Rajasekaran, S Narang, AA Zabreyko… - arXiv preprint arXiv …, 2024 - arxiv.org
We present MLTCP, a technique to augment today's congestion control algorithms to
accelerate DNN training jobs in shared GPU clusters. MLTCP enables the communication …
accelerate DNN training jobs in shared GPU clusters. MLTCP enables the communication …
DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
DistServe improves the performance of large language models (LLMs) serving by
disaggregating the prefill and decoding computation. Existing LLM serving systems colocate …
disaggregating the prefill and decoding computation. Existing LLM serving systems colocate …
Training DNN Models over Heterogeneous Clusters with Optimal Performance
C Nie, J Maghakian, Z Liu - arXiv preprint arXiv:2402.05302, 2024 - arxiv.org
Adjusting batch sizes and adaptively tuning other hyperparameters can significantly speed
up deep neural network (DNN) training. Despite the ubiquity of heterogeneous clusters …
up deep neural network (DNN) training. Despite the ubiquity of heterogeneous clusters …
A Codesign of Scheduling and Parallelization for Large Model Training in Heterogeneous Clusters
Joint consideration of scheduling and adaptive parallelism offers great opportunities for
improving the training efficiency of large models on heterogeneous GPU clusters. However …
improving the training efficiency of large models on heterogeneous GPU clusters. However …
Asymptotically Optimal Scheduling of Multiple Parallelizable Job Classes
Many modern computing workloads are composed of parallelizable jobs. A single
parallelizable job can be completed more quickly if it is run on additional servers, however …
parallelizable job can be completed more quickly if it is run on additional servers, however …