A finite-state Markov chain M can be regarded as a linear transform operating on the set of
probability distributions over its node set. The iterative applications of M to an initial …

Markov decision processes can be viewed as transformers of probability distributions. While
this view is useful from a practical standpoint to reason about trajectories of distributions …

We study distributional reachability for finite Markov decision processes (MDPs) from a
control theoretical perspective. Unlike standard probabilistic reachability notions, which are …

We consider distribution-based objectives for Markov Decision Processes (MDP). This class
of objectives gives rise to an interesting trade-off between full and partial information. As in …

This paper addresses a control problem for probabilistic models in the setting of Markov
decision processes (MDP). We are interested in the steady-state control problem which …

We study a parametric version of the Kannan-Lipton Orbit Problem for linear dynamical
systems. We show decidability in the case of one parameter and Skolem-hardness with two …

This work studies computation tree logic (CTL) model checking for finite-state Markov
decision processes (MDPs) over the space of their distributions. Instead of investigating …

We consider Markov decision processes (MDP) as generators of sequences of probability
distributions over states. A probability distribution is p-synchronizing if the probability mass is …

Markov Decision Processes (MDPs) are a classical model for decision making in the
presence of uncertainty. Often they are viewed as state transformers with planning …

Probabilistic model checking is a formal verification technique that has been applied
successfully in a variety of domains, providing identification of system errors through …