Distributed Machine Learning (DML) at the edge of the network involves model learning and inference across networking nodes over distributed data. One type of model learning could be the delivery of predictive analytics services to formulate intelligent environments, however, those environments heavily rely on real-time inference and are significantly influenced by changes in the underlying data (concept drifts). Moreover, the quality of service and availability in DML environments are directly tied to each node’s reliability, since such environments are highly susceptible to the impact of node failures. Even if such challenges can be tackled with distributed resilience mechanisms, their effectiveness and efficiency, due to concept drifts, should be maintained to ensure continuous and sustained quality of service. DML systems operate in dynamic environments, thus, they require their models to be updated according to the novel trends embedded in the new data they encounter. We, therefore, introduce several model maintenance mechanisms to ensure resilient DML systems in the long term when concept drifts emerge. We provide a comprehensive experimental evaluation of our resilience maintenance mechanisms over synthetic and real data showcasing their importance and applicability in edge learning environments.