Parameter-efficient fine-tuning for large models: A comprehensive survey

Z Han, C Gao, J Liu, SQ Zhang - arXiv preprint arXiv:2403.14608, 2024 - arxiv.org
Large models represent a groundbreaking advancement in multiple application fields,
enabling remarkable achievements across various tasks. However, their unprecedented …

Llamafactory: Unified efficient fine-tuning of 100+ language models

Y Zheng, R Zhang, J Zhang, Y Ye, Z Luo - arXiv preprint arXiv:2403.13372, 2024 - arxiv.org
Efficient fine-tuning is vital for adapting large language models (LLMs) to downstream tasks.
However, it requires non-trivial efforts to implement these methods on different models. We …

Trends and challenges of real-time learning in large language models: A critical review

M Jovanovic, P Voss - arXiv preprint arXiv:2404.18311, 2024 - arxiv.org
Real-time learning concerns the ability of learning systems to acquire knowledge over time,
enabling their adaptation and generalization to novel tasks. It is a critical ability for …

Navigating text-to-image customization: From lycoris fine-tuning to model evaluation

SY Yeh, YG Hsieh, Z Gao, BBW Yang… - The Twelfth …, 2023 - openreview.net
Text-to-image generative models have garnered immense attention for their ability to
produce high-fidelity images from text prompts. Among these, Stable Diffusion distinguishes …

Efficiency in focus: Layernorm as a catalyst for fine-tuning medical visual language pre-trained models

J Chen, D Yang, Y Jiang, M Li, J Wei, X Hou… - arXiv preprint arXiv …, 2024 - arxiv.org
In the realm of Medical Visual Language Models (Med-VLMs), the quest for universal
efficient fine-tuning mechanisms remains paramount, especially given researchers in …

Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

J Chen, Y Jiang, D Yang, M Li, J Wei, Z Qian… - arXiv preprint arXiv …, 2024 - arxiv.org
While large language models (LLMs) excel in world knowledge understanding, adapting
them to specific subfields requires precise adjustments. Due to the model's vast scale …

MING-MOE: Enhancing Medical Multi-Task Learning in Large Language Models with Sparse Mixture of Low-Rank Adapter Experts

Y Liao, S Jiang, Y Wang, Y Wang - arXiv preprint arXiv:2404.09027, 2024 - arxiv.org
Large language models like ChatGPT have shown substantial progress in natural language
understanding and generation, proving valuable across various disciplines, including the …

FLoRA: Low-Rank Core Space for N-dimension

C Si, X Wang, X Yang, Z Xu, Q Li, J Dai, Y Qiao… - arXiv preprint arXiv …, 2024 - arxiv.org
Adapting pre-trained foundation models for various downstream tasks has been prevalent in
artificial intelligence. Due to the vast number of tasks and high costs, adjusting all …

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning

T Jiang, S Huang, S Luo, Z Zhang, H Huang… - arXiv preprint arXiv …, 2024 - arxiv.org
Low-rank adaptation is a popular parameter-efficient fine-tuning method for large language
models. In this paper, we analyze the impact of low-rank updating, as implemented in LoRA …

Fine-tuning protein language models boosts predictions across diverse tasks

R Schmirler, M Heinzinger, B Rost - bioRxiv, 2023 - biorxiv.org
Prediction methods inputting embeddings from protein Language Models (pLMs) have
reached or even surpassed state-of-the-art (SOTA) performance on many protein prediction …