The average-reward formulation is a natural and important formulation of learning and
planning problems, yet has received much less attention than the episodic and discounted …