Distributed learning with compressed gradient differences

K Mishchenko, E Gorbunov, M Takáč… - … Methods and Software, 2024 - Taylor & Francis
Training large machine learning models requires a distributed computing approach, with
communication of the model updates being the bottleneck. For this reason, several methods …

Variance reduction is an antidote to byzantines: Better rates, weaker assumptions and communication compression as a cherry on the top

E Gorbunov, S Horváth, P Richtárik, G Gidel - arXiv preprint arXiv …, 2022 - arxiv.org
Byzantine-robustness has been gaining a lot of attention due to the growth of the interest in
collaborative and federated learning. However, many fruitful directions, such as the usage of …

Anomaly detection and defense techniques in federated learning: a comprehensive review

C Zhang, S Yang, L Mao, H Ning - Artificial Intelligence Review, 2024 - Springer
In recent years, deep learning methods based on a large amount of data have achieved
substantial success in numerous fields. However, with increases in regulations for protecting …

Communication compression for byzantine robust learning: New efficient algorithms and improved rates

A Rammal, K Gruntkowska, N Fedin… - International …, 2024 - proceedings.mlr.press
Byzantine robustness is an essential feature of algorithms for certain distributed optimization
problems, typically encountered in collaborative/federated learning. These problems are …

Byzantine robustness and partial participation can be achieved simultaneously: Just clip gradient differences

G Malinovsky, E Gorbunov, S Horváth… - Privacy Regulation and …, 2023 - openreview.net
Distributed learning has emerged as a leading paradigm for training large machine learning
models. However, in real-world scenarios, participants may be unreliable or malicious …

Partially personalized federated learning: Breaking the curse of data heterogeneity

K Mishchenko, R Islamov, E Gorbunov… - arXiv preprint arXiv …, 2023 - arxiv.org
We present a partially personalized formulation of Federated Learning (FL) that strikes a
balance between the flexibility of personalization and cooperativeness of global training. In …

Byzantine-tolerant methods for distributed variational inequalities

N Tupitsa, AJ Almansoori, Y Wu… - Advances in …, 2024 - proceedings.neurips.cc
Robustness to Byzantine attacks is a necessity for various distributed training scenarios.
When the training reduces to the process of solving a minimization problem, Byzantine …

Byzantine-robust distributed learning with compression

H Zhu, Q Ling - IEEE Transactions on Signal and Information …, 2023 - ieeexplore.ieee.org
Communication between workers and the master node to collect local stochastic gradients is
a key bottleneck in a large-scale distributed learning system. Various recent works have …

Communication-efficient and byzantine-robust distributed learning with error feedback

A Ghosh, RK Maity, S Kadhe… - IEEE Journal on …, 2021 - ieeexplore.ieee.org
We develop a communication-efficient distributed learning algorithm that is robust against
Byzantine worker machines. We propose and analyze a distributed gradient-descent …

Communication-efficient and byzantine-robust distributed learning

A Ghosh, RK Maity, S Kadhe… - 2020 Information …, 2020 - ieeexplore.ieee.org
We develop a communication-efficient distributed learning algorithm that is robust against
Byzantine worker machines. We propose and analyze a distributed gradient-descent …