Recently, Internet of Things (IoT) systems in the network edge with embedded intelligence emerged as a trending research topic. Edge computing offers a significant advantage over the traditional form of sharing personal data with a centralized entity since the latter paradigm may affect the user’s privacy, e.g., due to explicit exchange of sensitive biomedical data. To address this inherent data privacy issue, in this paper, we focus on designing an asynchronously weight updating federated learning algorithm toward the much anticipated AI-on-Edge IoT systems. Among numerous use-cases, we consider the face mask detection problem, which is traditionally considered as a centralized computer vision task. We take a different approach to distribute the learning tasks to the users in a federated learning framework, and then investigate the performance trade-off between synchronous and asynchronously weight updating methods. In our proposed system, the models are penalized by their performance metrics to limit a model’s participation in the aggregation stage. By developing the asynchronously weight updating method for deep learning (e.g., Convolutional Neural Network (CNN)) models, we also investigate its impact on model parameters exchange with the centralized aggregator. Experimental results demonstrate that our proposed asynchronously weight updating method achieves results comparable to those attained with the centralized training and the synchronously weight updating strategy. Also, we provide numerical analysis to demonstrate a significant transmission time overhead with our proposal.