Federated Low-Rank Adaptation of Foundation Models: Representation, Ownership, Transformation, and Cross-Round Continuity
Information · 10.11.2026 · 17 (11) · 1127
DOI: 10.3390/info17111127 ↗Kurz gefasst
Fachartikel in der Zeitschrift Information, online erschienen am 10.11.2026. Verfasst von Rongping Zhang und 6 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Low-rank adaptation (LoRA) reduces training and communication costs in federated fine-tuning of foundation models, including large language models (LLMs). However, a given effective weight update generally admits multiple low-rank factorizations. Shared states, private states, residuals, backbone models, and optimizer states may also be transformed during aggregation, refactorization, merging, and client re-entry. These transformations create state-consistency challenges across clients, rounds, and model versions that cannot be fully characterized by parameter averaging or communication volume alone. This survey reviews federated LoRA from a state-lifecycle perspective. It maps existing methods across state representation, ownership, transformation, cross-round continuity, and correctness, while comparing factor-based, effective weight-update, subspace, and composite representations, together with states used to support training and recovery. It also derives exact first-moment transport under fixed invertible factor reparameterization, analyzes the limits of diagonal second-moment migration, and distinguishes four outcomes of version changes: direct reuse, migration, reset, and rejection. The synthesis shows that trainable, uploaded, aggregated, and persistent states must be defined separately; aggregation in factor space is generally not equivalent to aggregation in weight-update space; and refactorization, backbone merging, and structural changes require explicit version management and state-migration rules. Building on these findings, this survey presents an evaluation checklist and research agenda for transformation fidelity, offline recovery, optimizer migration, and long-term storage, providing guidance for federated foundation-model adaptation across diverse deployment settings, including future sixth-generation (6G) edge-intelligence systems.
Zitieren
APA
Zhang, R., Wu, K., Yuan, X., Ni, W., Zhang, J. A., Liu, R. P., & Guo, Y. J. (2026). Federated Low-Rank Adaptation of Foundation Models: Representation, Ownership, Transformation, and Cross-Round Continuity. Information, 17(11), 1127. https://doi.org/10.3390/info17111127
Vancouver
Zhang R, Wu K, Yuan X, Ni W, Zhang JA, Liu RP, et al. Federated Low-Rank Adaptation of Foundation Models: Representation, Ownership, Transformation, and Cross-Round Continuity. Information. 2026;17(11):1127. doi:10.3390/info17111127
BibTeX
@article{Zhang2026,
author = {Zhang, Rongping and Wu, Kai and Yuan, Xin and Ni, Wei and Zhang, J. Andrew and Liu, Ren Ping and Guo, Y. Jay},
title = {Federated Low-Rank Adaptation of Foundation Models: Representation, Ownership, Transformation, and Cross-Round Continuity},
journal = {Information},
year = {2026},
volume = {17},
number = {11},
pages = {1127},
doi = {10.3390/info17111127}
}
EUVIMED