This paper surveys recent work by the author on the theoretical and algorithmic aspects of
restless bandit indexation as well as on its application to a variety of problems involving the …

This book provides a unified approach for the study of constrained Markov decision
processes with a finite state space and unbounded costs. Unlike the single controller case …

Heavy Traffic Analysis of Controlled Queueing and Communication Networks | SpringerLink
Skip to main content Advertisement SpringerLink Log in Menu Find a journal Publish with us …

We present in this chapter a survey on applications of MDPs to communication networks. We
survey both the different application areas in communication networks as well as the …

In this paper we develop the theory of constrained Markov games. We consider the expected
average cost as well as discounted cost. We allow different players to have different types of …

Consider a controlled Markov chain with countable state and action spaces. Basic quantities
that determine the values of average cost functionals are identified. Under some regularity …

Abstract Contents Introduction § 1. Controlled random sequences: main definitions and
traditional approaches § 1.1. Description of the mathematical model § 1.2. Models with …

This paper is the third in a series on constrained Markov decision processes (CMDPs) with a
countable state space and unbounded cost. In the previous papers we studied the expected …

The purpose of this paper is two fold. First to establish the theory of discounted constrained
Markov decision processes with a countable state and action spaces with general multi …

We present in this research report a survey on applications of MDPs to communication
networks. We survey both the different applications areas in communication networks as …