Industrial Control Systems (ICS) monitor and control physical processes. ICS are found in, among others, critical infrastructures such as water treatment plants, water distribution systems, and the electric power grid. While the existence of cyber-components in an ICS leads to ease of operations and maintenance, it renders the system under control vulnerable to cyber and physical attacks. An experimental study was conducted with replay attacks launched on an operational water distribution (WADI) plant to understand under what conditions an attacker/attack can remain undetected while stealing water. A detection method, based on an input-output Linear Time-invariant system model of the physical process, was developed and implemented in WADI to detect such attacks. The experiments reveal the strengths and limitations of the detection method and challenges faced by an attacker while attempting to steal water from a water distribution system.