In this paper, the distributed fault diagnosis (DFD) of networked dynamical systems with time-varying connected topologies, e.g., wireless sensor networks in harsh environments, is considered. Specifically, two essential problems are focused on, which are faced in extending the the decomposition-based adaptive DFD approach to such topology-varying systems. The problems introduced by the time-varying topologies are, respectively, decomposition schemes deterioration and pre-training difficulties. The causes of the two problems are detailed and addressed in our work. First, for the decomposition schemes deterioration problem, a multi-agent dynamics-based online distributed decomposition algorithm are developed, so that a decent decomposed network structure for such topology-varying network can be maintained. Second, to alleviate the pre-training difficulties in topology-varying systems, a fault detection method is proposed, which avoids the need for pre-training. The distributed decomposition algorithm is proved to converge in finite steps, and the proposed fault detection method is verified both theoretically and experimentally.