Cloud operators utilize collective communication optimizers to enhance the efficiency of the
single-tenant, centrally managed training clusters they manage. However, current optimizers …

Packet buffers in datacenter switches are shared across all the switch ports in order to
improve the overall throughput. The trend of shrinking buffer sizes in datacenter switches …

The all-to-all collective communications primitive is widely used in machine learning (ML)
and high performance computing (HPC) workloads, and optimizing its performance is of …

The surge of artificial intelligence, particularly large language models, has driven the rapid
development of large-scale machine learning clusters. Executing distributed models on …

Large-scale distributed training in production datacenters constitutes a challenging
workload bottlenecked by network communication. In response, both major industry players …

Financial exchanges have recently shown an interest in migrating to the public cloud for
scalability, elasticity, and cost savings. However, financial exchanges often have strict …

We show communication schedulers' recent work proposed for ML collectives does not
scale to the increasing problem sizes that arise from training larger models. These works …

As the scale and complexity of distributed workloads grows, performance is no longer the
sole objective sought by application developers and infrastructure operators, as they …