For humans, facial expressions are important social signals, and how we perceive specific individuals may be influenced by subtle emotional cues that they have given us in past encounters. A wide range of animal species are also capable of discriminating the emotions of others through facial expressions [1–5], and it is clear that remembering emotional experiences with specific individuals could have clear benefits for social bonding and aggression avoidance when these individuals are encountered again. Although there is evidence that non-human animals are capable of remembering the identity of individuals who have directly harmed them [6, 7], it is not known whether animals can form lasting memories of specific individuals simply by observing subtle emotional expressions that they exhibit on their faces. Here we conducted controlled experiments in which domestic horses were presented with a photograph of an angry or happy human face and several hours later saw the person who had given the expression in a neutral state. Short-term exposure to the facial expression was enough to generate clear differences in subsequent responses to that individual (but not to a different mismatched person), consistent with the past angry expression having been perceived negatively and the happy expression positively. Both humans were blind to the photograph that the horses had seen. Our results provide clear evidence that some non-human animals can effectively eavesdrop on the emotional state cues that humans reveal on a moment-to-moment basis, using their memory of these to guide future interactions with particular individuals.