In the age of artificial intelligence, the best approach to handling huge amounts of data is a
tremendously motivating and hard problem. Among machine learning models, stochastic …

Scalability and privacy are two critical concerns for cross-device federated learning (FL)
systems. In this work, we identify that synchronous FL–cannot scale efficiently beyond a few …

Federated learning enables resource-constrained edge compute devices, such as mobile
phones and IoT devices, to learn a shared model for prediction, while keeping the training …

Adaptive gradient methods that rely on scaling gradients down by the square root of
exponential moving averages of past squared gradients, such RMSProp, Adam, Adadelta …

We introduce a new and increasingly relevant setting for distributed optimization in machine
learning, where the data defining the optimization are unevenly distributed over an …

Modern large-scale machine learning applications require stochastic optimization
algorithms to be implemented on distributed computational architectures. A key bottleneck is …

Distributed training of deep learning models on large-scale training data is typically
conducted with asynchronous stochastic optimization to maximize the rate of updates, at the …

We study nonconvex finite-sum problems and analyze stochastic variance reduced gradient
(SVRG) methods for them. SVRG and related methods have recently surged into …

This paper deals with distributed finite-sum optimization for learning over multiple workers in
the presence of malicious Byzantine attacks. Most resilient approaches so far combine …

We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems,
where the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of …