This article aims to give a comprehensive and rigorous review of the principles and recent
development of coding for large-scale distributed machine learning (DML). With increasing …

A major bottleneck in distributed learning is the communication overhead of exchanging
intermediate model update parameters between the worker nodes and the parameter …

Communication overhead is one of the major performance bottlenecks in large-scale
distributed computing systems, in particular for machine learning applications …

Distributed learning has been widely adopted to train a global model from local data.
However, its performance can be severely affected by stragglers. Recently, some research …

Scaling up distributed machine learning systems face two major bottlenecks-delays due to
stragglers and limited communication bandwidth. Recently, a number of coding theoretic …

Codes are widely used in many engineering applications to offer robustness against noise.
In large-scale systems, there are several types of noise that can affect the performance of …

Distributed stochastic algorithms, equipped with gradient compression techniques, such as
codebook quantization, are becoming increasingly popular and considered state-of-the-art …

Coded distributed computation has become common practice for performing gradient
descent on large datasets to mitigate stragglers and other faults. This paper proposes a …

A distributed machine learning platform needs to recruit many heterogeneous worker nodes
to finish computation simultaneously. As a result, the overall performance may be degraded …

Edge computing has recently emerged as a promising paradigm to boost the performance of
distributed learning by leveraging the distributed resources at edge nodes. Architecturally …