Abstract Model fusion without accessing training data in machine learning has attracted
increasing interest due to the practical resource-saving and data privacy issues. During the …

Combining different models is a widely used paradigm in machine learning applications.
While the most common approach is to form an ensemble of models and average their …

The rapid expansion of the open-source language model landscape presents an opportunity
to merge the competencies of these model checkpoints by combining their parameters …

Deep model fusion/merging is an emerging technique that merges the parameters or
predictions of multiple deep learning models into a single one. It combines the abilities of …

Federated learning learns from scattered data by fusing collaborative models from local
nodes. However, due to chaotic information distribution, the model fusion may suffer from …

Low-Rank Adaptation (LoRA) is a popular technique for parameter-efficient fine-tuning of
Large Language Models (LLMs). We study how different LoRA modules can be merged to …

Incorporation of new knowledge into neural networks with simultaneous preservation of the
previous knowledge is known to be a nontrivial problem. This problem becomes even more …

Personalized federated learning (PFL) is a framework that targets individual models for
optimization, providing better privacy and flexibility for clients. However, in challenging …

OTFusion, or layer-wise model fusion via optimal transport, applies soft neuron association
to unify different pre-trained networks. Despite its effectiveness in saving computational …

Ensemble learning is a widespread technique to improve the prediction performance of
neural networks. However, it comes at the price of increased memory and inference time. In …