… To address this issue, we propose a data-free knowledge distillation method to fine-tune
the global model, so that the global model can preserve the knowledge in local models and …

… • We introduce an ensemble-based knowledge distillation technique to … on non-IID data
distribution. • We provide a generalized and simple method for federated knowledge distillation, …

… In the following, we start with the formulation of heterogeneous federated learning and
knowledge distillation. Then, we describe our critical learning strategies and the complete …

… non-IID data distribution across devices by building a local IID dataset on each device through
data … framework based on Two-step Knowledge Distillation, Fed2KD, has been proposed …

… of Non-IID data, they compromise the privacy of the raw data. … data augmentation strategy,
FedDA, to address the Non-IID … data reconstruction is regularized by knowledge distillation …

… knowledge in local clients, ensuring the sampled data is … by limited data and non-IID data
distributions by compensating for … classes using knowledge distillation from the global model. …

… Knowledge Distillation Given a teacher model T and a student model S, knowledge
distillation … To understand how the non-IID data affects federated learning, we performed an …

… that incorporate knowledge distillation into FL, we … data to facilitate the two-fold knowledge
distillation between local and global models to address the challenges posed by non-IID data …

… non-IID private data. For communication efficiency, we propose federated distillation (FD), a
distributed online knowledge distillation … Prior to operating FD, we rectify the non-IID training …

… essential to the success of the knowledge distillation as supported in [26]. Hence, we can
conclude that the intensity of non-IID data distributions exactly affects the model performances. …