Fedmfs: Federated multimodal fusion learning with selective modality communication
Federated learning (FL) is a distributed machine learning (ML) paradigm that enables clients
to collaborate without accessing, infringing upon, or leaking original user data by sharing …
to collaborate without accessing, infringing upon, or leaking original user data by sharing …
Communication-efficient multimodal federated learning: Joint modality and client selection
Multimodal federated learning (FL) aims to enrich model training in FL settings where clients
are collecting measurements across multiple modalities. However, key challenges to …
are collecting measurements across multiple modalities. However, key challenges to …
FedMEKT: Distillation-based Embedding Knowledge Transfer for Multimodal Federated Learning
Federated learning (FL) enables a decentralized machine learning paradigm for multiple
clients to collaboratively train a generalized global model without sharing their private data …
clients to collaboratively train a generalized global model without sharing their private data …
Fedmultimodal: A benchmark for multimodal federated learning
Over the past few years, Federated Learning (FL) has become an emerging machine
learning technique to tackle data privacy challenges through collaborative training. In the …
learning technique to tackle data privacy challenges through collaborative training. In the …
Multimodal federated learning via contrastive representation ensemble
With the increasing amount of multimedia data on modern mobile systems and IoT
infrastructures, harnessing these rich multimodal data without breaching user privacy …
infrastructures, harnessing these rich multimodal data without breaching user privacy …
Learn to combine modalities in multimodal deep learning
Combining complementary information from multiple modalities is intuitively appealing for
improving the performance of learning-based approaches. However, it is challenging to fully …
improving the performance of learning-based approaches. However, it is challenging to fully …
Enhancing Storage and Computational Efficiency in Federated Multimodal Learning for Large-Scale Models
Z Zhang, F Qi, C Xu - Forty-first International Conference on Machine … - openreview.net
The remarkable generalization of large-scale models has recently gained significant
attention in multimodal research. However, deploying heterogeneous large-scale models …
attention in multimodal research. However, deploying heterogeneous large-scale models …
Towards optimal multi-modal federated learning on non-IID data with hierarchical gradient blending
Recent advances in federated learning (FL) made it feasible to train a machine learning
model across multiple clients, even with non-IID data distributions. In contrast to these uni …
model across multiple clients, even with non-IID data distributions. In contrast to these uni …
MoPE: Parameter-Efficient and Scalable Multimodal Fusion via Mixture of Prompt Experts
R Jiang, L Liu, C Chen - arXiv preprint arXiv:2403.10568, 2024 - arxiv.org
Prompt-tuning has demonstrated parameter-efficiency in fusing unimodal foundation models
for multimodal tasks. However, its limited adaptivity and expressiveness lead to suboptimal …
for multimodal tasks. However, its limited adaptivity and expressiveness lead to suboptimal …
Harmony: Heterogeneous multi-modal federated learning through disentangled model training
Multi-modal sensing systems are increasingly prevalent in real-world applications such as
health monitoring and autonomous driving. Most multi-modal learning approaches need to …
health monitoring and autonomous driving. Most multi-modal learning approaches need to …