In the last few years, distributed machine learning has been usually executed over
heterogeneous networks such as a local area network within a multi-tenant cluster or a wide …

Decentralized SGD is an emerging training method for deep learning known for its much
less (thus faster) communication per iteration, which relaxes the averaging step in parallel …

The increasing size of machine learning models, especially deep neural network models,
can improve the model generalization capability. However, large models require more …

Decentralized learning has recently been attracting increasing attention for its applications
in parallel computation and privacy preservation. Many recent studies stated that the …

This article proposes a communication-efficient decentralized deep learning algorithm,
coined layer-wise federated group ADMM (L-FGADMM). To minimize an empirical risk …

Communication has been considered as a major bottleneck in large-scale decentralized
training systems since participating nodes iteratively exchange large amounts of …

Decentralized learning enables a group of collaborative agents to learn models using a
distributed dataset without the need for a central parameter server. Recently, decentralized …

Distributed deep learning (DL) has become prevalent in recent years to reduce training time
by leveraging multiple computing devices (eg, GPUs/TPUs) due to larger models and …

Decentralized learning provides an effective framework to train machine learning models
with data distributed over arbitrary communication graphs. However, most existing …

The current state-of-the-art decentralized learning algorithms mostly assume the data
distribution to be Independent and Identically Distributed (IID). However, in practical …