Federated Learning (FL) is a distributed learning paradigm that scales on-device learning
collaboratively and privately. Standard FL algorithms such as FEDAVG are primarily geared …

Stochastic gradient optimization is a class of widely used algorithms for training machine
learning models. To optimize an objective, it uses the noisy gradient computed from the …

Mini-batch optimization has proven to be a powerful paradigm for large-scale learning.
However, the state-of-the-art parallel mini-batch algorithms assume synchronous operation …

As a prevalent distributed learning paradigm, Federated Learning (FL) trains a global model
on a massive amount of devices with infrequent communication. This paper investigates a …

In this paper, we study the efficiency of a Restarted SubGradient (RSG) method that
periodically restarts the standard subgradient method (SG). We show that, when applied to a …

We study the problem of minimizing a strongly convex, smooth function when we have noisy
estimates of its gradient. We propose a novel multistage accelerated algorithm that is …

Recently there have been several attempts to extend Nesterov's accelerated algorithm to
smooth stochastic and variance-reduced optimization. In this paper, we show that there is a …

We study regularized stochastic convex optimization subject to linear equality constraints.
This class of problems was recently also studied by Ouyang et al.(2013) and Suzuki (2013); …

In this paper, we examine the convergence of mirror descent in a class of stochastic
optimization problems that are not necessarily convex (or even quasi-convex) and which we …

The extrapolation strategy raised by Nesterov, which can accelerate the convergence rate of
gradient descent methods by orders of magnitude when dealing with smooth convex …