The advent of Industry 4.0 has resulted in a significant increase in data availability, leading to the development and deployment of data-driven models for predicting the Remaining Useful Life (RUL) of machines. However, traditional centralized Predictive Maintenance (PreM) solutions that require complete access for training data give raise to concerns regarding data privacy. To address this challenge, Federated Learning (FL) has emerged as a promising and practical approach to enhance task performance while preserving data privacy within network nodes. Nevertheless, the presence of Non-Independent and Identically Distributed (Non-IID) data samples across devices can present challenges in terms of the convergence and speed of FL. Additionally, the heterogeneity of devices can lead to issues such as local model discarding and high communication costs, which are important considerations in FL. To address these challenges, this paper proposes Fedrated Predictive Maintenance (FedPreM), a novel federated reinforcement learning-based PreM scheme. FedPreM selectively involves a subset of devices in each communication round and employs an improved Perturbed Gradient Descent (PGD) optimizer to achieve flexible workload distribution among participating devices. By conducting experiments on a widely used turbofan dataset, our results demonstrate the effectiveness of FedPreM in reducing the number of communication rounds and minimizing prediction errors in distributed Industry 4.0 scenarios.