In Chap. 2, we formally introduce the concepts associated to a continuous time MDP.
Namely, the basic model of continuous-time MDPs and the concept of a Markov policy are …

In this paper we give conditions for (the existence and) several characterizations of
overtaking optimal policies for a general class of controlled diffusion processes. Our …

This book concerns continuous-time controlled Markov chains, also known as continuous-
time Markov decision processes. They form a class of stochastic control problems in which a …

This paper is a survey of recent results on continuous-time Markov decision processes
(MDPs) with unbounded transition rates, and reward rates that may be unbounded from …

En esta investigación se revisa la evolución teórica y práctica de los procesos markovianos
y se resalta su rápido avance y notorio potencial en el modelado de los procesos de toma …

This paper studies the impulse control of a general Markov process under the average (or
ergodic) cost when the impulse instants are restricted to be the arrival times of an …

This paper deals with continuous-time zero-sum two-person Markov games with
denumerable state space, general (Borel) action spaces and possibly unbounded transition …

This paper deals with unichain ergodic continuous-time controlled Markov chains (CMCs)
with a denumerable state space and possibly unbounded reward rates as well as …

We study the approximation of an ergodic average reward continuous-time denumerable
state Markov decision process (MDP) by means of a sequence of MDPs. Our results include …

This paper gives conditions for the convergence of the Laurent series expansion for a class
of continuous-time controlled Markov chains with possibly unbounded reward (or cost) rates …