Communication efficiency is crucial for accelerating distributed deep neural network (DNN)
training. All-reduce, a vital communication primitive, is responsible for reducing model …

Distributed deep training has become a significant consumer of bandwidth across
datacenter-scale networks. The diverse parallel strategies employed in deep training require …

From natural language processing to genome sequencing, large-scale machine learning
models are bringing advances to a broad range of fields. Many of these models are too large …

Abstract Parallel and distributed Deep Neural Network (DNN) training have become integral
in data centers, significantly reducing DNN training time. The interconnection type among …

Cloud and cluster computing platforms have become standard across almost every domain
of business, and their scale quickly approaches $\mathbf {O}(10^ 6) $ servers in a single …

As deep learning (DL) algorithms evolve and data volumes expand, training deep neural
networks (DNNs) has become essential across various domains, delivering unprecedented …

This thesis investigates demand-and topological reconfiguration-awareness in
communication networks on a macroscopic level along three dimensions. It presents …