The design and control of autonomous systems that operate in uncertain or adversarial
environments can be facilitated by formal modeling and analysis. Probabilistic model …

This position paper reflects on the state-of-the-art in decision-making under uncertainty. A
classical assumption is that probabilities can sufficiently capture all uncertainty in a system …

We present the probabilistic model checker Storm. Storm supports the analysis of discrete-
and continuous-time variants of both Markov chains and Markov decision processes. Storm …

We study safe policy improvement (SPI) for partially observable Markov decision processes
(POMDPs). SPI is an offline reinforcement learning (RL) problem that assumes access to (1) …

This paper focuses on inferring a general class of hidden Markov models (HMMs) using data
acquired from experts. Expert-acquired data contain decisions/actions made by …

We present a novel learning framework to obtain finite-state controllers (FSCs) for partially
observable Markov decision processes and illustrate its applicability for indefinite-horizon …

Uncertain partially observable Markov decision processes (uPOMDPs) allow the
probabilistic transition and observation functions of standard POMDPs to belong to a so …

Reinforcement Learning (RL) is a widely employed machine learning architecture that has
been applied to a variety of control problems. However, applications in safety-critical …

This paper marries two state-of-the-art controller synthesis methods for partially observable
Markov decision processes (POMDPs), a prominent model in sequential decision making …

Markov chain analysis is a key technique in formal verification. A practical obstacle is that all
probabilities in Markov models need to be known. However, system quantities such as …