AFLoRA: Adaptive LoRA for Resource-Efficient Federated Fine-Tuning of Large Language Models
IEEE Transactions on Mobile Computing · 2026-12 · 25 (12) · 24254-24266
DOI: 10.1109/tmc.2026.3732331 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2026-12. Verfasst von Yajie Zhou und 5 weiteren Autor:innen. Die Arbeit nennt 44 Literaturangaben. Gefördert von: R&D Program of Zhejiang, National Natural Science Foundation of China, Scientific Research Innovation Capability Support Project for Young Faculty.
Abstract (Originalsprache)
LoRA-based federated fine-tuning allows clients to fine-tune LoRA adapters locally and aggregates updates on the server, thereby enabling the efficient adaptation of global foundation models to distributed, domain-specific data. However, existing methods assume identical and static LoRA adapters across clients, which not only limits the utilization of heterogeneous resources and adaptation to diverse data distributions, but also prevents dynamic adjustment to clients’ evolving local fine-tuning state. They also struggle to achieve precise and lightweight aggregation of LoRA matrices, hindering learning performance in resource-constrained environments. To address these issues, we propose AFLoRA, an adaptive LoRA-based resource-efficient federated fine-tuning framework. For effectively adapting to Non-IID local data while minimizing resource consumption, AFLoRA introduces diagonal-based dynamic rank assignment, which dynamically adjusts local LoRA ranks based on client resources and local adaptation state guided by a learnable diagonal matrix. It further decouples LoRA matrices updates between clients and the server to reduce overhead, and develops a zero-padding-based rank-aware aggregation mechanism to accurately integrate heterogeneous updates through lossless matrix expansion and rank-aware weighting. These mechanisms enable AFLoRA to balance resource efficiency and fine-tuning performance effectively. Extensive experiments demonstrate that AFLoRA outperforms state-of-the-art methods in both LLM adaptation performance and fine-tuning efficiency in heterogeneous environments.
Förderung
- R&D Program of Zhejiang
- National Natural Science Foundation of China
- Scientific Research Innovation Capability Support Project for Young Faculty
- Shandong Provincial Natural Science Foundation
- Ganpo Talent
- Jiangxi Provincial Natural Science Foundation
- Youth Talent Training Program for Early-Career Professionals
Zitieren
APA
Zhou, Y., Pang, X., Wang, Z., Hu, J., Jin, S., & Ren, K. (2026). AFLoRA: Adaptive LoRA for Resource-Efficient Federated Fine-Tuning of Large Language Models. IEEE Transactions on Mobile Computing, 25(12), 24254-24266. https://doi.org/10.1109/tmc.2026.3732331
Vancouver
Zhou Y, Pang X, Wang Z, Hu J, Jin S, Ren K. AFLoRA: Adaptive LoRA for Resource-Efficient Federated Fine-Tuning of Large Language Models. IEEE Transactions on Mobile Computing. 2026;25(12):24254-24266. doi:10.1109/tmc.2026.3732331
BibTeX
@article{Zhou2026,
author = {Zhou, Yajie and Pang, Xiaoyi and Wang, Zhibo and Hu, Jiahui and Jin, Shuaifan and Ren, Kui},
title = {AFLoRA: Adaptive LoRA for Resource-Efficient Federated Fine-Tuning of Large Language Models},
journal = {IEEE Transactions on Mobile Computing},
year = {2026},
volume = {25},
number = {12},
pages = {24254--24266},
doi = {10.1109/tmc.2026.3732331}
}
EUVIMED