FedSpaLLM: Federated Pruning of Large Language Models

G Bai, Y Li, Z Li, L Zhao, K Kim - arXiv preprint arXiv:2410.14852, 2024 - arxiv.org
Large Language Models (LLMs) achieve state-of-the-art performance but are challenging to
deploy due to their high computational and storage demands. Pruning can reduce model …

FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction

F Wu, X Wang, Y Wang, T Liu, L Su, J Gao - arXiv preprint arXiv …, 2024 - arxiv.org
In federated learning (FL), accommodating clients' varied computational capacities poses a
challenge, often limiting the participation of those with constrained resources in global …

Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models

J Luo, C Chen, S Wu - arXiv preprint arXiv:2410.10114, 2024 - arxiv.org
Prompt learning for pre-trained Vision-Language Models (VLMs) like CLIP has
demonstrated potent applicability across diverse downstream tasks. This lightweight …

Model Pruning-enabled Federated Split Learning for Resource-constrained Devices in Artificial Intelligence Empowered Edge Computing Environment

Y Jia, B Liu, X Zhang, F Dai, A Khan, L Qi… - ACM Transactions on …, 2024 - dl.acm.org
Distributed Collaborative Machine Learning (DCML) has emerged in artificial intelligence-
empowered edge computing environments, such as the Industrial Internet of Things (IIoT), to …

Improving Generalization and Personalization in Model-Heterogeneous Federated Learning

X Zhang, J Wang, W Bao, Y Zhang… - … on Neural Networks …, 2024 - ieeexplore.ieee.org
Conventional federated learning (FL) assumes the homogeneity of models, necessitating
clients to expose their model parameters to enhance the performance of the server model …

Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learning

K Yi, T Kharisov, I Sokolov, P Richtárik - arXiv preprint arXiv:2406.01115, 2024 - arxiv.org
Virtually all federated learning (FL) methods, including FedAvg, operate in the following
manner: i) an orchestrating server sends the current model parameters to a cohort of clients …

Advances in Robust Federated Learning: Heterogeneity Considerations

C Chen, T Liao, X Deng, Z Wu, S Huang… - arXiv preprint arXiv …, 2024 - arxiv.org
In the field of heterogeneous federated learning (FL), the key challenge is to efficiently and
collaboratively train models across multiple clients with different data distributions, model …