Networking has become a well-known performance bottleneck for distributed machine learning (DML). Although lots of works have focused on accelerating the communication …
Sparse tensors appear frequently in distributed deep learning, either as a direct artifact of the deep neural network's gradients, or as a result of an explicit sparsification process …
W Zhao, X Jiao, M Hu, X Li, X Zhang… - 2022 IEEE International …, 2022 - ieeexplore.ieee.org
Click-through rate (CTR) prediction is one of the most crucial components in the online advertising industry. In order to produce a personalized CTR prediction, an industry-level …
H Wang, Y Qin, CL Lao, Y Le, W Wu… - 2023 IEEE 31st …, 2023 - ieeexplore.ieee.org
Recent works introduce In-Network Aggregation (INA) for distributed training (DT), which moves the gradient summation into network programmable switches. INA can reduce the …
J Fang, G Zhao, H Xu, Z Yu, B Shen… - 2023 IEEE/ACM 31st …, 2023 - ieeexplore.ieee.org
The surging scale of distributed training (DT) incurs significant communication overhead in datacenters, while a promising solution is in-network aggregation (INA). It leverages …
W Ning, H Sun, X Fu, X Yang, Q Qi… - IEEE Journal on …, 2021 - ieeexplore.ieee.org
Distributed edge learning collaborates powerful edge devices to train a shared global model. Since the frequent communication between the server and workers is very …
As deep neural networks (DNNs) grow in complexity and size, the resultant increase in communication overhead during distributed training has become a significant bottleneck …
Z Zhang, Q Qi, R Shang, L Chen, F Xu - Proceedings of the 50th …, 2021 - dl.acm.org
Optimizing performance for Distributed Deep Neural Network (DDNN) training has recently become increasingly compelling, as the DNN model gets complex and the training dataset …