Stochastic Approximation with Delayed Updates: Finite-Time Rates under Markovian Sampling

A Adibi, N Dal Fabbro, L Schenato… - International …, 2024 - proceedings.mlr.press
Motivated by applications in large-scale and multi-agent reinforcement learning, we study
the non-asymptotic performance of stochastic approximation (SA) schemes with delayed …

Federated offline reinforcement learning: Collaborative single-policy coverage suffices

J Woo, L Shi, G Joshi, Y Chi - arXiv preprint arXiv:2402.05876, 2024 - arxiv.org
Offline reinforcement learning (RL), which seeks to learn an optimal policy using offline data,
has garnered significant interest due to its potential in critical applications where online data …

Compressed Federated Reinforcement Learning with a Generative Model

A Beikmohammadi, S Khirirat, S Magnússon - arXiv preprint arXiv …, 2024 - arxiv.org
Reinforcement learning has recently gained unprecedented popularity, yet it still grapples
with sample inefficiency. Addressing this challenge, federated reinforcement learning …

DASA: Delay-Adaptive Multi-Agent Stochastic Approximation

ND Fabbro, A Adibi, HV Poor, SR Kulkarni… - arXiv preprint arXiv …, 2024 - arxiv.org
We consider a setting in which $ N $ agents aim to speedup a common Stochastic
Approximation (SA) problem by acting in parallel and communicating with a central server …

Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environments

H Wang, S He, Z Zhang, F Miao, J Anderson - arXiv preprint arXiv …, 2024 - arxiv.org
We explore a Federated Reinforcement Learning (FRL) problem where $ N $ agents
collaboratively learn a common policy without sharing their trajectory data. To date, existing …

A Single Online Agent Can Efficiently Learn Mean Field Games

C Zhang, X Chen, X Di - arXiv preprint arXiv:2405.03718, 2024 - arxiv.org
Mean field games (MFGs) are a promising framework for modeling the behavior of large-
population systems. However, solving MFGs can be challenging due to the coupling of …

One-Shot Averaging for Distributed TD (λ) Under Markov Sampling

H Tian, IC Paschalidis… - IEEE Control Systems …, 2024 - ieeexplore.ieee.org
We consider a distributed setup for reinforcement learning, where each agent has a copy of
the same Markov Decision Process but transitions are sampled from the corresponding …