Federated Fine-Tuning for Pre-Trained Foundation Models Over Wireless Networks
IEEE Transactions on Wireless Communications · 2025-04 · 24 (4) · 3450-3464
DOI: 10.1109/twc.2025.3531128 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Wireless Communications, 2025-04. Verfasst von Zixin Wang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 62-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Shanghai, National Natural Science Foundation of China, Shanghai Rising-Star Program.
Abstract (Originalsprache)
Pre-trained foundation models (FMs), with extensive number of neurons, are key to advancing next-generation intelligence services, where personalizing these models requires massive amount of task-specific data and computational resources. The prevalent solution involves centralized processing at the edge server, which, however, raises privacy concerns due to the transmission of raw data. Instead, federated fine-tuning (FedFT) is an emerging privacy-preserving fine-tuning (FT) paradigm for personalized pre-trained foundation models. In particular, by integrating low-rank adaptation (LoRA) with federated learning (FL), federated LoRA enables the collaborative FT of a global model with edge devices, achieving comparable learning performance to full FT while training fewer parameters over distributed data and preserving raw data privacy. However, the limited radio resources and computation capabilities of edge devices pose significant challenges for deploying 3 LoRA over wireless networks. To this paper, we propose a split federated LoRA framework, which deploys the computationally-intensive encoder of a pre-trained model at the edge server, while keeping the embedding and task modules at the edge devices. The information exchanges between these modules occur over wireless networks. Building on this split framework, the paper provides a rigorous analysis of the upper bound of the convergence gap for the wireless federated LoRA system. This analysis reveals the weighted impact of the number of edge devices participating in FedFT over all rounds, motivating the formulation of a long-term upper bound minimization problem. To address the long-term constraint, we decompose the formulated long-term mixed-integer programming (MIP) problem into sequential sub-problems using the Lyapunov technique. We then develop an online algorithm for effective device scheduling and bandwidth allocation. Simulation results demonstrate the effectiveness of the proposed online algorithm in enhancing learning performance.
Förderung
- Natural Science Foundation of Shanghai
- National Natural Science Foundation of China
- Shanghai Rising-Star Program
- Hong Kong Research Grants Council
- Areas of Excellence Scheme
Zitieren
APA
Wang, Z., Zhou, Y., Shi, Y., & Letaief, K. B. (2025). Federated Fine-Tuning for Pre-Trained Foundation Models Over Wireless Networks. IEEE Transactions on Wireless Communications, 24(4), 3450-3464. https://doi.org/10.1109/twc.2025.3531128
Vancouver
Wang Z, Zhou Y, Shi Y, Letaief KB. Federated Fine-Tuning for Pre-Trained Foundation Models Over Wireless Networks. IEEE Transactions on Wireless Communications. 2025;24(4):3450-3464. doi:10.1109/twc.2025.3531128
BibTeX
@article{Wang2025,
author = {Wang, Zixin and Zhou, Yong and Shi, Yuanming and Letaief, Khaled B.},
title = {Federated Fine-Tuning for Pre-Trained Foundation Models Over Wireless Networks},
journal = {IEEE Transactions on Wireless Communications},
year = {2025},
volume = {24},
number = {4},
pages = {3450--3464},
doi = {10.1109/twc.2025.3531128}
}
EUVIMED