Peta-scale embedded photonics architecture for distributed deep learning applications
As Deep Learning (DL) models grow larger and more complex, training jobs are
increasingly distributed across multiple Computing Units (CU) such as GPUs and TPUs …
increasingly distributed across multiple Computing Units (CU) such as GPUs and TPUs …
Impact of RoCE congestion control policies on distributed training of DNNs
Ahstract-RDMA over Converged Ethernet (RoCE) has gained significant attraction for
datacenter networks due to its compatibility with conventional Ethernet-based fabric …
datacenter networks due to its compatibility with conventional Ethernet-based fabric …
TACOS: Topology-Aware Collective Algorithm Synthesizer for Distributed Machine Learning
W Won, M Elavazhagan, S Srinivasan, A Durg… - arXiv preprint arXiv …, 2023 - arxiv.org
The surge of artificial intelligence, specifically large language models, has led to a rapid
advent towards the development of large-scale machine learning training clusters …
advent towards the development of large-scale machine learning training clusters …