The theory of Markov decision processes focuses on controlled Markov chains in discrete
time. The authors establish the theory for general state and action spaces and at the same …

We propose a theoretical framework for approximate planning and learning in partially
observed systems. Our framework is based on the fundamental notion of information state …

We consider the simultaneous seat-inventory control of a set of parallel flights between a
common origin and destination with dynamic customer choice among the flights. We …

In recent years, various metrics have been developed for measuring the behavioral similarity
of states in probabilistic transition systems [J. Desharnais et al., Proceedings of …

Many sequential decision problems can be formulated as Markov decision processes
(MDPs) where the optimal value function (or cost-to-go function) can be shown to satisfy a …

In Markov models of sequential decision processes, one is often interested in showing that
the value function is monotonic, convex, and/or supermodular in the state variables. These …

This paper proposes a simple analytical model called M time scale Markov decision process
(MMDPs) for hierarchically structured sequential decision making processes, where …

We present tools and guidelines for investigating Lipschitz continuity of the value functions
in MDP's, using the Hausdorff metric and the Kantorovich metric for measuring the influence …

The economics of search is a prominent component of economic theory, and it has a
richness and elegance that underpins a host of practical applications. In this book Brian and …

The traditional approach to POMDPs is to convert them into fully observed MDPs by
considering a belief state as an information state. However, a belief-state based approach …