Is Q-learning minimax optimal? a tight sample complexity analysis
Q-learning, which seeks to learn the optimal Q-function of a Markov decision process (MDP)
in a model-free fashion, lies at the heart of reinforcement learning. When it comes to the …
in a model-free fashion, lies at the heart of reinforcement learning. When it comes to the …
The efficacy of pessimism in asynchronous Q-learning
This paper is concerned with the asynchronous form of Q-learning, which applies a
stochastic approximation scheme to Markovian data samples. Motivated by the recent …
stochastic approximation scheme to Markovian data samples. Motivated by the recent …
Breaking the sample complexity barrier to regret-optimal model-free reinforcement learning
Achieving sample efficiency in online episodic reinforcement learning (RL) requires
optimally balancing exploration and exploitation. When it comes to a finite-horizon episodic …
optimally balancing exploration and exploitation. When it comes to a finite-horizon episodic …
Sample complexity of asynchronous Q-learning: Sharper analysis and variance reduction
Asynchronous Q-learning aims to learn the optimal action-value function (or Q-function) of a
Markov decision process (MDP), based on a single trajectory of Markovian samples induced …
Markov decision process (MDP), based on a single trajectory of Markovian samples induced …
Ucb momentum q-learning: Correcting the bias without forgetting
Abstract We propose UCBMQ, Upper Confidence Bound Momentum Q-learning, a new
algorithm for reinforcement learning in tabular and possibly stage-dependent, episodic …
algorithm for reinforcement learning in tabular and possibly stage-dependent, episodic …
Sample complexity of asynchronous Q-learning: Sharper analysis and variance reduction
Asynchronous Q-learning aims to learn the optimal action-value function (or Q-function) of a
Markov decision process (MDP), based on a single trajectory of Markovian samples induced …
Markov decision process (MDP), based on a single trajectory of Markovian samples induced …
Finite-time analysis for double Q-learning
Although Q-learning is one of the most successful algorithms for finding the best action-
value function (and thus the optimal policy) in reinforcement learning, its implementation …
value function (and thus the optimal policy) in reinforcement learning, its implementation …
Online target q-learning with reverse experience replay: Efficiently finding the optimal policy for linear mdps
Q-learning is a popular Reinforcement Learning (RL) algorithm which is widely used in
practice with function approximation (Mnih et al., 2015). In contrast, existing theoretical …
practice with function approximation (Mnih et al., 2015). In contrast, existing theoretical …
Accelerating value iteration with anchoring
Value Iteration (VI) is foundational to the theory and practice of modern reinforcement
learning, and it is known to converge at a $\mathcal {O}(\gamma^ k) $-rate. Surprisingly …
learning, and it is known to converge at a $\mathcal {O}(\gamma^ k) $-rate. Surprisingly …
Breaking the sample complexity barrier to regret-optimal model-free reinforcement learning
Achieving sample efficiency in online episodic reinforcement learning (RL) requires
optimally balancing exploration and exploitation. When it comes to a finite-horizon episodic …
optimally balancing exploration and exploitation. When it comes to a finite-horizon episodic …