Improved communication efficiency in federated natural policy gradient via admm-based gradient updates

G Lan, H Wang, J Anderson, C Brinton… - arXiv preprint arXiv …, 2023 - arxiv.org
Federated reinforcement learning (FedRL) enables agents to collaboratively train a global
policy without sharing their individual data. However, high communication overhead …

Model-free learning with heterogeneous dynamical systems: A federated LQR approach

H Wang, LF Toso, A Mitra, J Anderson - arXiv preprint arXiv:2308.11743, 2023 - arxiv.org
We study a model-free federated linear quadratic regulator (LQR) problem where M agents
with unknown, distinct yet similar dynamics collaboratively learn an optimal policy to …

Finite-time analysis of on-policy heterogeneous federated reinforcement learning

C Zhang, H Wang, A Mitra, J Anderson - arXiv preprint arXiv:2401.15273, 2024 - arxiv.org
Federated reinforcement learning (FRL) has emerged as a promising paradigm for reducing
the sample complexity of reinforcement learning tasks by exploiting information from …

Stochastic Approximation with Delayed Updates: Finite-Time Rates under Markovian Sampling

A Adibi, N Dal Fabbro, L Schenato… - International …, 2024 - proceedings.mlr.press
Motivated by applications in large-scale and multi-agent reinforcement learning, we study
the non-asymptotic performance of stochastic approximation (SA) schemes with delayed …

Asynchronous federated reinforcement learning with policy gradient updates: Algorithm design and convergence analysis

G Lan, DJ Han, A Hashemi, V Aggarwal… - arXiv preprint arXiv …, 2024 - arxiv.org
To improve the efficiency of reinforcement learning, we propose a novel asynchronous
federated reinforcement learning framework termed AFedPG, which constructs a global …

Federated td learning over finite-rate erasure channels: Linear speedup under markovian sampling

N Dal Fabbro, A Mitra… - IEEE Control Systems …, 2023 - ieeexplore.ieee.org
Federated learning (FL) has recently gained much attention due to its effectiveness in
speeding up supervised learning tasks under communication and privacy constraints …

Federated offline reinforcement learning: Collaborative single-policy coverage suffices

J Woo, L Shi, G Joshi, Y Chi - arXiv preprint arXiv:2402.05876, 2024 - arxiv.org
Offline reinforcement learning (RL), which seeks to learn an optimal policy using offline data,
has garnered significant interest due to its potential in critical applications where online data …

Compressed federated reinforcement learning with a generative model

A Beikmohammadi, S Khirirat, S Magnússon - Joint European Conference …, 2024 - Springer
Reinforcement learning has recently gained unprecedented popularity, yet it still grapples
with sample inefficiency. Addressing this challenge, federated reinforcement learning …

Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environments

H Wang, S He, Z Zhang, F Miao, J Anderson - arXiv preprint arXiv …, 2024 - arxiv.org
We explore a Federated Reinforcement Learning (FRL) problem where $ N $ agents
collaboratively learn a common policy without sharing their trajectory data. To date, existing …

The Sample-Communication Complexity Trade-off in Federated Q-Learning

S Salgia, Y Chi - arXiv preprint arXiv:2408.16981, 2024 - arxiv.org
We consider the problem of federated Q-learning, where $ M $ agents aim to collaboratively
learn the optimal Q-function of an unknown infinite-horizon Markov decision process with …