As deep neural networks (DNNs) grow in complexity and size, the resultant increase in communication overhead during distributed training has become a significant bottleneck …
Quantization is a fundamental optimization for many machine-learning use cases, including compressing gradients, model weights and activations, and datasets. The most accurate …
Gradient aggregation has long been identified as a major bottleneck in today's large-scale distributed machine learning training systems. One promising solution to mitigate such …
Z Zhang, H Shen - arXiv preprint arXiv:2408.04107, 2024 - arxiv.org
In large-language models, memory constraints in the key-value cache (KVC) pose a challenge during inference, especially with long prompts. In this work, we observed that …
Distributed Mean Estimation (DME), in which $ n $ clients communicate vectors to a parameter server that estimates their average, is a fundamental building block in …
Software-defined networking enables tight integration between packet-processing hardware and centralized controllers, highlighting the importance of deep network insight for informed …
Abstract Machine learning (ML) has become ubiquitous, integrating into numerous real-life applications. However, meeting the computational demands of ML systems is challenging …