It is well understood that client-master communication can be a primary bottleneck in
Federated Learning. In this work, we address this issue with a novel client subsampling …

Nowadays, the training data and neural network models are getting increasingly large. The
training time of deep learning will become unbearably long on a single machine. To reduce …

We present a unified framework for analyzing local SGD methods in the convex and strongly
convex regimes for distributed/federated training of supervised machine learning models …

Deep learning have achieved promising results on a wide spectrum of AI applications.
Larger datasets and models consistently yield better performance. However, we generally …

Modern large-scale machine learning applications require stochastic optimization
algorithms to be implemented on distributed compute systems. A key bottleneck of such …

Many communication-efficient variants of SGD use gradient quantization schemes. These
schemes are often heuristic and fixed over the course of training. We empirically observe …

Many compression techniques have been proposed to reduce the communication overhead
of Federated Learning training procedures. However, these are typically designed for …

Federated Learning (FL) is a distributed machine learning approach where multiple clients
work together to solve a machine learning task. One of the key challenges in FL is the issue …

Abstract We present Rotated Adaptive Tetra-iterated Quantizer (RATQ), afixed-length
quantizer for gradients in first order stochasticoptimization. RATQ is easy to implement and …

Training deep neural networks on large datasets can often be accelerated by using multiple
compute nodes. This approach, known as distributed training, can utilize hundreds of …