EF21: A new, simpler, theoretically better, and practically faster error feedback
P Richtárik, I Sokolov… - Advances in Neural …, 2021 - proceedings.neurips.cc
Error feedback (EF), also known as error compensation, is an immensely popular
convergence stabilization mechanism in the context of distributed training of supervised …
convergence stabilization mechanism in the context of distributed training of supervised …
Rethinking gradient sparsification as total error minimization
Gradient compression is a widely-established remedy to tackle the communication
bottleneck in distributed training of large deep neural networks (DNNs). Under the error …
bottleneck in distributed training of large deep neural networks (DNNs). Under the error …
Permutation compressors for provably faster distributed nonconvex optimization
We study the MARINA method of Gorbunov et al (2021)--the current state-of-the-art
distributed non-convex optimization method in terms of theoretical communication …
distributed non-convex optimization method in terms of theoretical communication …
Basis matters: better communication-efficient second order methods for federated learning
Recent advances in distributed optimization have shown that Newton-type methods with
proper communication compression mechanisms can guarantee fast local rates and low …
proper communication compression mechanisms can guarantee fast local rates and low …
Personalized federated learning with multiple known clusters
We consider the problem of personalized federated learning when there are known cluster
structures within users. An intuitive approach would be to regularize the parameters so that …
structures within users. An intuitive approach would be to regularize the parameters so that …
Catalyst Acceleration of Error Compensated Methods Leads to Better Communication Complexity
Communication overhead is well known to be a key bottleneck in large scale distributed
learning, and a particularly successful class of methods which help to overcome this …
learning, and a particularly successful class of methods which help to overcome this …
Deep-ultraviolet optoelectronic devices enabled by the hybrid integration of next-generation semiconductors and emerging device platforms
N Alfaraj - 2019 - repository.kaust.edu.sa
In this dissertation, the design and fabrication of deep-ultraviolet photodetectors were
investigated based on gallium oxide and its alloys, through the heterogeneous integration …
investigated based on gallium oxide and its alloys, through the heterogeneous integration …
[PDF][PDF] Non-convex Stochastic Optimization With Biased Gradient Estimators
I Sokolov - 2022 - repository.kaust.edu.sa
Non-convex optimization problems appear in various applications of machine learning.
Because of their practical importance, these problems gained a lot of attention in recent …
Because of their practical importance, these problems gained a lot of attention in recent …
[引用][C] Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learning
K KAUST