Heterogeneous lora for federated fine-tuning of on-device foundation models
Foundation models (FMs) adapt surprisingly well to downstream tasks with fine-tuning.
However, their colossal parameter space prohibits their training on resource-constrained …
However, their colossal parameter space prohibits their training on resource-constrained …
FedLFC: Towards Efficient Federated Multilingual Modeling with LoRA-based Language Family Clustering
Abstract Federated Multilingual Modeling (FMM) plays a crucial role in the applications of
natural language processing due to the increasing diversity of languages and the growing …
natural language processing due to the increasing diversity of languages and the growing …
Heterogeneous Low-Rank Approximation for Federated Fine-tuning of On-Device Foundation Models
Large foundation models (FMs) adapt surprisingly well to specific domains or tasks with fine-
tuning. Federated learning (FL) further enables private FM fine-tuning using the local data …
tuning. Federated learning (FL) further enables private FM fine-tuning using the local data …
Synergizing Foundation Models and Federated Learning: A Survey
The recent development of Foundation Models (FMs), represented by large language
models, vision transformers, and multimodal models, has been making a significant impact …
models, vision transformers, and multimodal models, has been making a significant impact …
FedHLT: Efficient Federated Low-Rank Adaption with Hierarchical Language Tree for Multilingual Modeling
Federated Multilingual Modeling (FMM) has become an essential approach in natural
language processing (NLP) due to increasing linguistic diversity and the heightened …
language processing (NLP) due to increasing linguistic diversity and the heightened …