In recent years, deep artificial neural networks (including recurrent ones) have won
numerous contests in pattern recognition and machine learning. This historical survey …

Sequence prediction and classification are ubiquitous and challenging problems in machine
learning that can require identifying complex dependencies between temporally distant …

Learning to store information over extended time intervals by recurrent backpropagation
takes a very long time, mostly because of insufficient, decaying error backflow. We briefly …

Recurrent networks (crossreference Chapter 12) can, in principle, use their feedback
connections to store representations of recent input events in the form of activations. The …

Abstract Lifelong Machine Learning, or LML, considers systems that can learn many tasks
from one or more domains over its lifetime. The goal is to sequentially retain learned …

Continual learning is the constant development of complex behaviors with no final end in
mind. It is the process of learning ever more complicated skills by building on those skills …

In algorithmic music composition, a simple technique involves selecting notes sequentially
according to a transition table that specifies the probability of the next note as a function of …

Acquire the tools for understanding new architectures and algorithms of dynamical recurrent
networks (DRNs) from this valuable field guide, which documents recent forays into artificial …

Continual learning is the constant development of increasingly complex behaviors; the
process of building more complicated skills on top of those already developed. A continual …

This paper addresses the general problem of reinforcement learning (RL) in partially
observable environments. In 2013, our large RL recurrent neural networks (RNNs) learned …