Abstract Running Stochastic Gradient Descent (SGD) in a decentralized fashion has shown
promising results. In this paper we propose Moniqua, a technique that allows decentralized …

Distributed stochastic gradient descent (SGD) is essential for scaling the machine learning
algorithms to a large number of computing nodes. However, the infrastructures variability …

Due to the explosion in the size of the training datasets, distributed learning has received
growing interest in recent years. One of the major bottlenecks is the large communication …

We focus on the commonly used synchronous Gradient Descent paradigm for large-scale
distributed learning, for which there has been a growing interest to develop efficient and …

Communication bottleneck has been identified as a significant issue in distributed
optimization of large-scale learning models. Recently, several approaches to mitigate this …

This paper focuses on communication-efficient federated learning problem, and develops a
novel distributed quantized gradient approach, which is characterized by adaptive …

Big data, including applications with high security requirements, are often collected and
stored on multiple heterogeneous devices, such as mobile devices, drones, and vehicles …

To accelerate the training of machine learning models, distributed stochastic gradient
descent (SGD) and its variants have been widely adopted, which apply multiple workers in …

The alternating direction method of multipliers (ADMM) algorithm has been widely employed
for distributed machine learning tasks. However, it suffers from several limitations, eg, a …