[PDF][PDF] Dynamic programming principles for learning MFCs

H Gu, X Guo, X Wei, R Xu - arXiv preprint arXiv:1911.07314, 2019 - researchgate.net
This paper establishes the time consistent property, ie, the dynamic programming principle
(DPP), for learning mean field controls (MFCs). The key idea is to define the correct form of …

Dynamic programming principles for mean-field controls with learning

H Gu, X Guo, X Wei, R Xu - arXiv preprint arXiv:1911.07314, 2019 - arxiv.org
Dynamic programming principle (DPP) is fundamental for control and optimization, including
Markov decision problems (MDPs), reinforcement learning (RL), and more recently mean …

Dynamic programming principles for mean-field controls with learning

H Gu, X Guo, X Wei, R Xu - Operations Research, 2023 - pubsonline.informs.org
The dynamic programming principle (DPP) is fundamental for control and optimization,
including Markov decision problems (MDPs), reinforcement learning (RL), and, more …

[PDF][PDF] Q-learning for mean-field controls

H Gu, X Guo, X Wei, R Xu - arXiv preprint arXiv:2002.04131, 2020 - researchgate.net
Multi-agent reinforcement learning (MARL) has been applied to many challenging problems
including two-team computer games, autonomous drivings, and real-time biddings. Despite …

Model-free mean-field reinforcement learning: mean-field MDP and mean-field Q-learning

R Carmona, M Laurière, Z Tan - The Annals of Applied Probability, 2023 - projecteuclid.org
We study infinite horizon discounted mean field control (MFC) problems with common noise
through the lens of mean field Markov decision processes (MFMDP). We allow the agents to …

Unified reinforcement Q-learning for mean field game and control problems

A Angiuli, JP Fouque, M Laurière - Mathematics of Control, Signals, and …, 2022 - Springer
Abstract We present a Reinforcement Learning (RL) algorithm to solve infinite horizon
asymptotic Mean Field Game (MFG) and Mean Field Control (MFC) problems. Our approach …

Convergence of Multi-Scale Reinforcement Q-Learning Algorithms for Mean Field Game and Control Problems

A Angiuli, JP Fouque, M Laurière, M Zhang - arXiv preprint arXiv …, 2023 - arxiv.org
We establish the convergence of the unified two-timescale Reinforcement Learning (RL)
algorithm presented by Angiuli et al. This algorithm provides solutions to Mean Field Game …

Actor-Critic learning for mean-field control in continuous time

N Frikha, M Germain, M Laurière, H Pham… - arXiv preprint arXiv …, 2023 - arxiv.org
We study policy gradient for mean-field control in continuous time in a reinforcement
learning setting. By considering randomised policies with entropy regularisation, we derive a …

Continuous-time q-learning for McKean-Vlasov control problems

X Wei, X Yu - arXiv preprint arXiv:2306.16208, 2023 - arxiv.org
This paper studies the q-learning, recently coined as the continuous-time counterpart of Q-
learning by Jia and Zhou (2022c), for continuous time Mckean-Vlasov control problems in …

Mean-field control based approximation of multi-agent reinforcement learning in presence of a non-decomposable shared global state

WU Mondal, V Aggarwal, SV Ukkusuri - arXiv preprint arXiv:2301.06889, 2023 - arxiv.org
Mean Field Control (MFC) is a powerful approximation tool to solve large-scale Multi-Agent
Reinforcement Learning (MARL) problems. However, the success of MFC relies on the …