It is known from both theory and experiments that introducing time delays into the communication network of mobile-agent swarms produces coherent rotational patterns. Often such spatiotemporal rotations can be bistable with other swarming patterns, such as milling and flocking. Yet, most known bifurcation results related to delay-coupled swarms rely on inaccurate mean-field techniques. As a consequence, the utility of applying macroscopic theory as a guide for predicting and controlling swarms of mobile robots has been limited. To overcome this limitation, we perform an exact stability analysis of two primary swarming patterns in a general model with time-delayed interactions. By correctly identifying the relevant spatio-temporal modes that determine stability in the presence of time delay, we are able to accurately predict bistability and unstable oscillations in large swarm simulations–laying the groundwork for comparisons to robotics experiments.