Leaks in water distribution systems can run continuously for extended periods undetected due to their minimal impact on pressure and vibration signals in the overall system. Detecting such leaks from acoustic measurements is challenging because leak-induced changes in acoustic measurements can be masked by strong background noise or usage-induced changes. This paper addressed the problem of leak detection and localization in water distribution pipes through a technique called linear prediction (LP). LP was shown to be effective in capturing the composite spectrum effects of radiation, pipe system, and leak-induced excitation of the pipe system, with and without leaks, and thus has the potential to be an effective tool to detect leaks. The relatively simple mathematical formulation of LP lends itself well to online implementation in long-term monitoring applications and hence motivated an in-depth investigation. A data-driven anomaly detection approach was presented which utilizes the features extracted from the LP coefficients representing the underlying acoustic signals. In terms of leak localization, compared with correlation techniques using raw signals, it was shown that shorter segments of LP reconstructed signals can achieve similar levels of accuracy as those using longer segments of raw time series, which is a key advantage in long-term online implementation applications. A relatively complex experimental test bed was used to generate realistic acoustic data under various hydraulic conditions, including simulated leak and flow cases. Most importantly, due to the simplicity of the technique, this method has significant potential for autonomous leak detection and localization in full-scale monitoring applications.