Building Communication Efficient Asynchronous Peer-to-Peer Federated LLMs with Blockchain
Proceedings of the AAAI Symposium Series · 20.05.2024 · 3 (1) · 288-292
DOI: 10.1609/aaaiss.v3i1.31212 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 20.05.2024. Verfasst von Sree Bhargavi Balija und 2 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref).
Abstract (Originalsprache)
Large language models (LLM) have gathered attention with the advent of ChatGPT. However, developing personalized LLM models faces challenges in real-world applications due to data scarcity and privacy concerns. Federated learning addresses these issues, providing collaborative training while preserving the client’s data. Although it has made significant progress, federated learning still faces ongoing challenges, such as communication efficiency, heterogeneous data, and privacy-preserving methods. This paper presents a novel, fully decentralized federated learning framework for LLMs to address these challenges. We utilize different blockchain-federated LLM (BC-FL) algorithms, effectively balancing the trade-off between latency and accuracy in a decentralized-federated learning environment. Additionally, we address the challenge of communication overhead in peer-to-peer networks by optimizing the path for weight transfer and mitigating node anomalies. We conducted experiments to evaluate memory usage and latency in server and serverless environments. Our results demonstrate a decrease in latency by 5X and a 13% increase in accuracy for serverless cases. Comparisons between synchronous and asynchronous scenarios revealed a 76% reduction in information passing time for the latter. The PageRank method is most efficient in eliminating anomalous nodes for better performance of the global federated LLM model. The code is available on GitHub (https://github.com/Sreebhargavibalijaa/Federated_finetuning_LLM-s_p2p_environment)
Zitieren
APA
Balija, S. B., Nanda, A., & Sahoo, D. (2024). Building Communication Efficient Asynchronous Peer-to-Peer Federated LLMs with Blockchain. Proceedings of the AAAI Symposium Series, 3(1), 288-292. https://doi.org/10.1609/aaaiss.v3i1.31212
Vancouver
Balija SB, Nanda A, Sahoo D. Building Communication Efficient Asynchronous Peer-to-Peer Federated LLMs with Blockchain. Proceedings of the AAAI Symposium Series. 2024;3(1):288-292. doi:10.1609/aaaiss.v3i1.31212
BibTeX
@article{Balija2024,
author = {Balija, Sree Bhargavi and Nanda, Amitash and Sahoo, Debashis},
title = {Building Communication Efficient Asynchronous Peer-to-Peer Federated LLMs with Blockchain},
journal = {Proceedings of the AAAI Symposium Series},
year = {2024},
volume = {3},
number = {1},
pages = {288--292},
doi = {10.1609/aaaiss.v3i1.31212}
}
EUVIMED