… The aforementioned limitation of FL has motivated to the development of federated distillation
(FD)[… Finally, to transfer the downloaded global knowledge into local models, each worker …

… model using aggregated knowledge from heterogeneous users, … Moreover, the ensemble
knowledge is not fully utilized to … In this work, we propose a data-free knowledge distillation …

… the knowledge from previous rounds, and the local training induces forgetting the knowledge
… To this end, we propose a novel and effective algorithm, Federated Not-True Distillation (…

… For all the CV tasks, we set the channels of each group as 16. The optimizer of client training
used is SGD, we tune the learning rate in {0.1, 0.05, 0.01} for FedAvg and set the learning …

… of existing federated distillation methods, we in this paper propose a novel federated distillation …
training with a proximal term in the model objective [16]. FEDDFUSION is a data-based …

… (2) for one class, the importance of knowledge are different among local models of clients.
To facilitate more effective knowledge distillation, we propose customized label sampling and …

… knowledge distillation process (FedDKD). In FedDKD, we introduce a decentralized knowledge
distillation (DKD) module to distill the knowledge of local … speed of the algorithms [16]. …

… We compare FedPAQ with 16-bit or 8-bit precision levels. … for personalized learning
and knowledge distillation. In addition, … -efficient than other compared federated learning-based …

… Motivated by the paradigm of Knowledge Distillation (KD) [12–16] that knowledge can be
transferred from a neural network to another via exchanging soft predictions instead of using …

… distillation algorithm that aggregates structural knowledge with … inherent heterogeneity of
decentralized federated learning. … address this issue by distilling on public data [16, 22, 4], they …