UAV surveillance and tracking have attracted great enthusiasm in intelligent transportation, and various approaches have been reported up to now. However, these approaches often ignored the uncertainties in the urban environment, such as occlusion, view change, and background clutter. Ignoring these uncertain factors often leads to a reduction in surveillance performance and tracking quality. This study devotes to improving the cooperative surveillance capability of multi-UAV formation by designing different cooperative strategies in the urban environment. To be specific, a novel cooperative architecture is designed to control the observation locations of multiple UAVs throughout the formation process. For different types of interference, we introduce a novel target recognition rate of each UAV as the decision factor and design corresponding cooperative strategies to guarantee the accuracy of cooperative surveillance. Based on this architecture, we develop a vision-based method of cooperative surveillance and tracking by multiple UAVs (SMART) whose objective function is the motion cost and flight reliability of UAVs to ensure that each UAV can be in the optimal surveillance location for the target. The proposed SMART skillfully integrates the strict, elastic, and flight constraint strategies. During the execution of the multi-UAV formation, the inherent safety constraints of multiple UAVs and the designed strategies are used to solve the quadratic optimization model to adjust the locations of these UAVs. To demonstrate the superiority of our method, we conduct a 3D simulation urban environment and devise several experiments to analyze the performance of SMART on it. The experimental results demonstrate that SMART can not only maintain the high cooperative flight capability, but also provide high flexibility and fault tolerance.