ATLAS: Adaptive Task-Aware Federated Learning with Heterogeneous LoRA and Split Learning for Edge LLMs
Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 1068-1075
DOI: 10.1609/aaaiss.v10i1.43528 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Mahmoud S. Mayaleh und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Federated learning (FL) can adapt large language models (LLMs) without centralizing private data, but deployment on heterogeneous edge clients is constrained by task diversity, device variability, and communication cost. Standard task-agnostic aggregation and uniform-rank low-rank adaptation (LoRA) often cause negative transfer and poor resource matching. This paper presents ATLAS, a task-aware framework for parameter-efficient federated fine-tuning of transformer LLMs on heterogeneous clients. ATLAS combines representation-based task discovery, budget-aware heterogeneous LoRA allocation, state-consistent split federated learning, and LoRA-only graph regularization, together with INT8 LoRA communication and FP32 server-side computation. On a heterogeneous GLUE federation with three tasks, nine clients, and three backbones, ATLAS improves the macroaverage task score over standard FedAvg by 16.18 points on DistilBERT, 24.83 points on GPT-2, and 6.86 points on Qwen-2.5-0.5B, while reducing LoRA synchronization traffic and eliminating low-resource-client failures.
Zitieren
APA
Mayaleh, M. S., Houda, Z. A. E., & Khoukhi, L. (2026). ATLAS: Adaptive Task-Aware Federated Learning with Heterogeneous LoRA and Split Learning for Edge LLMs. Proceedings of the AAAI Symposium Series, 10(1), 1068-1075. https://doi.org/10.1609/aaaiss.v10i1.43528
Vancouver
Mayaleh MS, Houda ZAE, Khoukhi L. ATLAS: Adaptive Task-Aware Federated Learning with Heterogeneous LoRA and Split Learning for Edge LLMs. Proceedings of the AAAI Symposium Series. 2026;10(1):1068-1075. doi:10.1609/aaaiss.v10i1.43528
BibTeX
@article{Mayaleh2026,
author = {Mayaleh, Mahmoud S. and Houda, Zakaria Abou El and Khoukhi, Lyes},
title = {ATLAS: Adaptive Task-Aware Federated Learning with Heterogeneous LoRA and Split Learning for Edge LLMs},
journal = {Proceedings of the AAAI Symposium Series},
year = {2026},
volume = {10},
number = {1},
pages = {1068--1075},
doi = {10.1609/aaaiss.v10i1.43528}
}
EUVIMED