EUVIMEDEUVIMED entdecken
Crossref

Tri-AFLLM: Resource-Efficient Adaptive Asynchronous Accelerated Federated LLMs

Dewen Qiao, Xiang Ao, Yu Liu, Xuetao Chen, Fuyuan Song, Zheng Qin, Wenqiang Jin

IEEE Transactions on Circuits and Systems for Video Technology · 2025-05 · 35 (5) · 4198-4211

DOI: 10.1109/tcsvt.2024.3519790 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-05. Verfasst von Dewen Qiao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 47 Literaturangaben. Gefördert von: National Natural Science Foundation of China, National Key Research and Development Projects.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The local deployment of federated large language models (FLLM) has further advanced the development of edge intelligence. However, the resource constraints of end devices, device heterogeneity, and the non-independent and identically distributed (Non-IID) nature of data pose significant challenges to the application of FLLM. To address this issue, we propose an Adaptive Asynchronous Accelerated FLLM (Tri-AFLLM) algorithm to achieve the efficient utilization of limited resources and improve model accuracy in the edge computing (EC) scenarios. Specifically, Tri-AFLLM first ships an off-the-shelf LLM, i.e., CLIP, to each end device, keeping the backbone parameters frozen and updating only the parameters of the adapter containing two linear transformation layers by using momentum gradient descent (MGD). Next, a toy example is provided to illustrate the necessity of using different numbers of local iterations for heterogeneous devices in resource-constrained environments. Subsequently, the convergence bound of the Tri-AFLLM under a given resource budget is discussed. Then, we formulated the bound into a resource consumption minimization problem with the number of local iterations as the optimization variable under a given model accuracy to mitigate the contribution disparity of local models to the global aggregation. Finally, extensive experiments are conducted to validate the superiority of Tri-AFLLM in terms of resource consumption, model accuracy, and addressing the Non-IID problem.

Förderung

  • National Natural Science Foundation of China
  • National Key Research and Development Projects

Zitieren

APA

Qiao, D., Ao, X., Liu, Y., Chen, X., Song, F., Qin, Z., & Jin, W. (2025). Tri-AFLLM: Resource-Efficient Adaptive Asynchronous Accelerated Federated LLMs. IEEE Transactions on Circuits and Systems for Video Technology, 35(5), 4198-4211. https://doi.org/10.1109/tcsvt.2024.3519790

Vancouver

Qiao D, Ao X, Liu Y, Chen X, Song F, Qin Z, et al. Tri-AFLLM: Resource-Efficient Adaptive Asynchronous Accelerated Federated LLMs. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(5):4198-4211. doi:10.1109/tcsvt.2024.3519790

BibTeX
@article{Qiao2025,
  author = {Qiao, Dewen and Ao, Xiang and Liu, Yu and Chen, Xuetao and Song, Fuyuan and Qin, Zheng and Jin, Wenqiang},
  title = {Tri-AFLLM: Resource-Efficient Adaptive Asynchronous Accelerated Federated LLMs},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {5},
  pages = {4198--4211},
  doi = {10.1109/tcsvt.2024.3519790}
}
Kostenlos starten →