Resource management in computing is a very challenging problem that involves making
sequential decisions. Resource limitations, resource heterogeneity, dynamic and diverse …

Nowadays, the training data and neural network models are getting increasingly large. The
training time of deep learning will become unbearably long on a single machine. To reduce …

Efficient collective communication is crucial to parallel-computing applications such as
distributed training of large-scale recommendation systems and natural language …

Gradient compression is a widely-established remedy to tackle the communication
bottleneck in distributed training of large deep neural networks (DNNs). Under the error …

Federated learning (FL) is becoming a popular paradigm for collaborative learning over
distributed, private datasets owned by non-trusting entities. FL has seen successful …

Proximal splitting algorithms are well suited to solving large-scale nonsmooth optimization
problems, in particular those arising in machine learning. We propose a new primal-dual …

Gradient compression is a promising approach to alleviating the communication bottleneck
in data parallel deep neural network (DNN) training by significantly reducing the data …

In distributed or federated optimization and learning, communication between the different
computing units is often the bottleneck and gradient compression is widely used to reduce …

Sparse tensors appear frequently in federated deep learning, either as a direct artifact of the
deep neural network's gradients, or as a result of an explicit sparsification process. Existing …

Federated learning (FL) enables distributed model training from local data collected by
users. In distributed systems with constrained resources and potentially high dynamics, eg …