Many memory models suggest self-terminating backward scanning along a memory representation. In these models, time to retrieve a particular item from memory could depend on how far in the past the item was presented or on the number of items presented since that item. To investigate which of these two types of memory representation is more likely, we designed a relative Judgment of Recency (JOR) task with variable presentation rates. The variable presentation rate deconfounded the age of memory and the number of intervening items. Our results favor the hypothesis that memory representation is temporally organized. This result is important for advancing memory models and for building stronger ties between cognitive and neural models of memory.