In hierarchical cosmological models, galaxies grow in mass through the continual accretion of smaller ones. The tidal disruption of these systems is expected to result in loosely bound stars surrounding the galaxy, at distances that reach 10–100 times the radius of the central disk^,. The number, luminosity and morphology of the relics of this process provide significant clues to galaxy formation history, but obtaining a comprehensive survey of these components is difficult because of their intrinsic faintness and vast extent. Here we report a panoramic survey of the Andromeda galaxy (M31). We detect stars and coherent structures that are almost certainly remnants of dwarf galaxies destroyed by the tidal field of M31. An improved census of their surviving counterparts implies that three-quarters of M31’s satellites brighter than M_v = -6 await discovery. The brightest companion, Triangulum (M33), is surrounded by a stellar structure that provides persuasive evidence for a recent encounter with M31. This panorama of galaxy structure directly confirms the basic tenets of the hierarchical galaxy formation model and reveals the shared history of M31 and M33 in the unceasing build-up of galaxies.