The wide proliferation of drones has posed great threats to personal privacy and public security, which makes it urgent to monitor the drones in the sensitive areas. Drone surveillance using acoustic arrays, as a promising method, could localize and track the drones effectively. In this paper, we develop a systematic method for drone localization and tracking by using acoustic arrays. Specifically, we develop a time difference of arrival (TDOA) estimation algorithm based on Gauss priori probability density function to overcome the multipath effect and the low signal-to-noise ratio (SNR), and design a localization method by making full use of the TDOA estimation results, followed by tracking the drone by Kalman filter. Finally, field experiments are carried out to verify the effectiveness and performance of the proposed methods. The results show that they can achieve good performance in drone localization and tracking.