In this article, we aim to provide a literature review of different formulations and approaches
to continual reinforcement learning (RL), also known as lifelong or non-stationary RL. We …

Highlights•Multiple scales of learning (and hence meta-learning) are ubiquitous in
nature.•Many existing lines of work in neuroscience and cognitive science touch upon …

The discovery of functional molecules is an expensive and time-consuming process,
exemplified by the rising costs of small molecule therapeutic discovery. One class of …

General-purpose robotic systems must master a large repertoire of diverse skills to be useful
in a range of daily tasks. While reinforcement learning provides a powerful framework for …

Trading off exploration and exploitation in an unknown environment is key to maximising
expected return during learning. A Bayes-optimal policy, which does so optimally, conditions …

Foundation models have shown impressive adaptation and scalability in supervised and self-
supervised learning problems, but so far these successes have not fully translated to …

Currently, it is hard to reap the benefits of deep learning for Bayesian methods, which allow
the explicit specification of prior knowledge and accurately capture model uncertainty. We …

Foundation models have shown impressive adaptation and scalability in supervised and self-
supervised learning problems, but so far these successes have not fully translated to …

Modern machine learning requires system designers to specify aspects of the learning
pipeline, such as losses, architectures, and optimizers. Meta-learning, or learning-to-learn …

Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a
transition model. Here, we propose a new hypothesis: that replay is able to implement a form …