This chapter shows how visitors draw connections between historical and contemporary migration in relation to what they encounter in museum displays. It focuses on a small-scale research project conducted over the summer of 2015 with long-term residents whom people invited to visit the permanent display, Destination Tyneside, at Discovery Museum in Newcastle, UK. The chapter highlights how these research participants utilised not only resources such as museum objects and interpretive materials, but also childhood memories, debates in the media and narratives about place, in order to make sense of and review their position towards migrants. It explores the different ways in which the participants responded to the museum's invitation to empathise, and investigates how affective responses are part of these experiences. The chapter looks at the ways in which visitors negotiate empathy in the exhibition space through different kinds of 'memory work' provides a hitherto-underexplored way to understand seemingly complex and contradictory visitor responses to migration displays.