A survey of distributed optimization
In distributed optimization of multi-agent systems, agents cooperate to minimize a global
function which is a sum of local objective functions. Motivated by applications including …
function which is a sum of local objective functions. Motivated by applications including …
Distributed optimization for control
Advances in wired and wireless technology have necessitated the development of theory,
models, and tools to cope with the new challenges posed by large-scale control and …
models, and tools to cope with the new challenges posed by large-scale control and …
Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent
Most distributed machine learning systems nowadays, including TensorFlow and CNTK, are
built in a centralized fashion. One bottleneck of centralized algorithms lies on high …
built in a centralized fashion. One bottleneck of centralized algorithms lies on high …
Network topology and communication-computation tradeoffs in decentralized optimization
In decentralized optimization, nodes cooperate to minimize an overall objective function that
is the sum (or average) of per-node private objective functions. Algorithms interleave local …
is the sum (or average) of per-node private objective functions. Algorithms interleave local …
Asynchronous decentralized parallel stochastic gradient descent
Most commonly used distributed machine learning systems are either synchronous or
centralized asynchronous. Synchronous algorithms like AllReduce-SGD perform poorly in a …
centralized asynchronous. Synchronous algorithms like AllReduce-SGD perform poorly in a …
Next: In-network nonconvex optimization
P Di Lorenzo, G Scutari - IEEE Transactions on Signal and …, 2016 - ieeexplore.ieee.org
We study nonconvex distributed optimization in multiagent networks with time-varying
(nonsymmetric) connectivity. We introduce the first algorithmic framework for the distributed …
(nonsymmetric) connectivity. We introduce the first algorithmic framework for the distributed …
Distributed stochastic gradient tracking methods
In this paper, we study the problem of distributed multi-agent optimization over a network,
where each agent possesses a local cost function that is smooth and strongly convex. The …
where each agent possesses a local cost function that is smooth and strongly convex. The …
Adaptation, learning, and optimization over networks
AH Sayed - Foundations and Trends® in Machine Learning, 2014 - nowpublishers.com
This work deals with the topic of information processing over graphs. The presentation is
largely self-contained and covers results that relate to the analysis and design of multi-agent …
largely self-contained and covers results that relate to the analysis and design of multi-agent …
Distributed optimization over time-varying directed graphs
A Nedić, A Olshevsky - IEEE Transactions on Automatic Control, 2014 - ieeexplore.ieee.org
We consider distributed optimization by a collection of nodes, each having access to its own
convex function, whose collective goal is to minimize the sum of the functions. The …
convex function, whose collective goal is to minimize the sum of the functions. The …
Distributed continuous-time optimization: nonuniform gradient gains, finite-time convergence, and convex constraint set
In this paper, a distributed optimization problem with general differentiable convex objective
functions is studied for continuous-time multi-agent systems with single-integrator dynamics …
functions is studied for continuous-time multi-agent systems with single-integrator dynamics …