Worldwide Federated Training of Language Models

A Iacob, L Sani, B Marino, P Aleksandrov… - arXiv preprint arXiv …, 2024 - arxiv.org
The reliance of language model training on massive amounts of computation and vast
datasets scraped from potentially low-quality, copyrighted, or sensitive data has come into …

PISTOL: Dataset Compilation Pipeline for Structural Unlearning of LLMs

X Qiu, WF Shen, Y Chen, N Cancedda… - arXiv preprint arXiv …, 2024 - arxiv.org
Recently, machine unlearning, which seeks to erase specific data stored in the pre-trained
or fine-tuned models, has emerged as a crucial protective measure for LLMs. However …

Sheaf HyperNetworks for Personalized Federated Learning

B Nguyen, L Sani, X Qiu, P Liò, ND Lane - arXiv preprint arXiv:2405.20882, 2024 - arxiv.org
Graph hypernetworks (GHNs), constructed by combining graph neural networks (GNNs) with
hypernetworks (HNs), leverage relational data across various domains such as neural …

FedSheafHN: Personalized Federated Learning on Graph-structured Data

W Liang, Y Zhao, R She, Y Li, WP Tay - arXiv preprint arXiv:2405.16056, 2024 - arxiv.org
Personalized subgraph Federated Learning (FL) is a task that customizes Graph Neural
Networks (GNNs) to individual client needs, accommodating diverse data distributions …

A deep cut into Split Federated Self-supervised Learning

M Przewięźlikowski, M Osial, B Zieliński… - arXiv preprint arXiv …, 2024 - arxiv.org
Collaborative self-supervised learning has recently become feasible in highly distributed
environments by dividing the network layers between client devices and a central server …

Advances in Robust Federated Learning: Heterogeneity Considerations

C Chen, T Liao, X Deng, Z Wu, S Huang… - arXiv preprint arXiv …, 2024 - arxiv.org
In the field of heterogeneous federated learning (FL), the key challenge is to efficiently and
collaboratively train models across multiple clients with different data distributions, model …

FedMAP: Unlocking Potential in Personalized Federated Learning through Bi-Level MAP Optimization

F Zhang, C Esteve-Yagüe, S Dittmer… - arXiv preprint arXiv …, 2024 - arxiv.org
Federated Learning (FL) enables collaborative training of machine learning models on
decentralized data while preserving data privacy. However, data across clients often differs …