Distributed learning with compressed gradient differences
Training large machine learning models requires a distributed computing approach, with
communication of the model updates being the bottleneck. For this reason, several methods …
communication of the model updates being the bottleneck. For this reason, several methods …
Distributed optimization with arbitrary local solvers
With the growth of data and necessity for distributed optimization methods, solvers that work
well on a single machine must be re-designed to leverage distributed computation. Recent …
well on a single machine must be re-designed to leverage distributed computation. Recent …
Stochastic, distributed and federated optimization for machine learning
J Konečný - arXiv preprint arXiv:1707.01155, 2017 - arxiv.org
We study optimization algorithms for the finite sum problems frequently arising in machine
learning applications. First, we propose novel variants of stochastic gradient descent with a …
learning applications. First, we propose novel variants of stochastic gradient descent with a …
Communication trade-offs for synchronized distributed SGD with large step size
KK Patel, A Dieuleveut - arXiv preprint arXiv:1904.11325, 2019 - arxiv.org
Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning.
However, in practice, its convergence is bottlenecked by slow communication rounds …
However, in practice, its convergence is bottlenecked by slow communication rounds …
Clustering with distributed data
S Kar, B Swenson - arXiv preprint arXiv:1901.00214, 2019 - arxiv.org
We consider $ K $-means clustering in networked environments (eg, internet of things (IoT)
and sensor networks) where data is inherently distributed across nodes and processing …
and sensor networks) where data is inherently distributed across nodes and processing …
Communication optimality trade-offs for distributed estimation
AK Sahu, D Jakovetic, S Kar - arXiv preprint arXiv:1801.04050, 2018 - arxiv.org
This paper proposes $\mathbf {C} $ ommunication efficient $\mathbf {RE} $ cursive $\mathbf
{D} $ istributed estimati $\mathbf {O} $ n algorithm, $\mathcal {CREDO} $, for networked …
{D} $ istributed estimati $\mathbf {O} $ n algorithm, $\mathcal {CREDO} $, for networked …
Communication efficient distributed weighted non-linear least squares estimation
The paper addresses design and analysis of communication-efficient distributed algorithms
for solving weighted non-linear least squares problems in multi-agent networks …
for solving weighted non-linear least squares problems in multi-agent networks …
Communication efficient distributed learning with feature partitioned data
One major bottleneck in the design of large scale distributed machine learning algorithms is
the communication cost. In this paper, we propose and analyze a distributed learning …
the communication cost. In this paper, we propose and analyze a distributed learning …
On Seven Fundamental Optimization Challenges in Machine Learning
K Mishchenko - arXiv preprint arXiv:2110.12281, 2021 - arxiv.org
Many recent successes of machine learning went hand in hand with advances in
optimization. The exchange of ideas between these fields has worked both ways, with …
optimization. The exchange of ideas between these fields has worked both ways, with …
[PDF][PDF] Communication trade-offs for synchronized distributed SGD (Local-SGD) with large step size (with Appendix)
A DIEULEVEUT, KK PATEL - proceedings.neurips.cc
Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning.
However, in practice, its convergence is bottlenecked by slow communication rounds …
However, in practice, its convergence is bottlenecked by slow communication rounds …