One obstacle to applying reinforcement learning algorithms to real-world problems is the
lack of suitable reward functions. Designing such reward functions is difficult in part because …

The development of Artificial General Intelligence (AGI) promises to be a major event. Along
with its many potential benefits, it also raises serious safety concerns (Bostrom, 2014). The …

Abstract Theory of mind (ToM) broadly refers to humans' ability to represent the mental
states of others, including their desires, beliefs, and intentions. We design a Theory of Mind …

We present a suite of reinforcement learning environments illustrating various safety
properties of intelligent agents. These problems include safe interruptibility, avoiding side …

Modern AI systems have become of widespread use in almost all sectors with a strong
impact on our society. However, the very methods on which they rely, based on Machine …

Bayesian reward learning from demonstrations enables rigorous safety and uncertainty
analysis when performing imitation learning. However, Bayesian reward learning methods …

As AI systems are integrated into high stakes social domains, researchers now examine how
to design and operate them in a safe and ethical manner. However, the criteria for identifying …

To analyze the expected behavior of advanced artificial agents, we consider a formal
idealized agent that makes observations that inform it about its goal, and we find that it can …

Can humans get arbitrarily capable reinforcement learning (RL) agents to do their bidding?
Or will sufficiently capable RL agents always find ways to bypass their intended objectives …

Inverse reinforcement learning (IRL) attempts to infer human rewards or preferences from
observed behavior. Since human planning systematically deviates from rationality, several …