Street-to-Shop fashion item retrieval is an instance-level image retrieval task in which a photo from a user is used to query a fashion image database in order to retrieve either the same or similar fashion items. This task is particularly challenging due to the domain shift between database photos, which tend to be stages, professional shots, and consumer photos that have a much greater variety in terms of quality, pose, etc. To reduce the problem difficulty, state-of-the-art approaches train one retrieval model per domain or fashion item category. In this work we propose a single detect-then-retrieve model that can be applied to any (query or database) image and which outperforms methods using domain or category-specific retrieval models by significant margins on the Exact Street2Shop benchmark dataset.