EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Beyond Single-Framework Architectures: A Systematic Evaluation and Hybrid Design for Scalable Multi-Agent Coordination

Muhammad Saleem Khan, Muhammad Janas Khan, Muhammad Sharif, Sadaf Yasmin

IEEE Access · 2026 · 14 · 59688-59705

DOI: 10.1109/access.2026.3683900 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Muhammad Saleem Khan und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language model (LLM)-based multi-agent frameworks have emerged as powerful tools for complex coordination tasks, yet systematic comparisons across realistic, large-scale scenarios remain limited. This paper presents a comprehensive evaluation of three prominent frameworks, i.e., CrewAI, AutoGen, and LangChain, alongside a novel hybrid architecture that strategically combines their complementary strengths. Using CREW-WILDFIRE, a realistic wildfire response benchmark featuring heterogeneous agents and partial observability, we evaluated the framework’s performance across 17 task levels and 7 behavioral competencies. Our results reveal distinct architectural trade-offs: CrewAI achieves 34% higher task designation scores through hierarchical coordination but lacks adaptability; AutoGen demonstrates 28% superior plan adaptation via conversational feedback, yet incurs high token costs; LangChain offers maximum flexibility at the expense of a 42% increase in computational overhead. Based on these insights, we propose and validate a hybrid LangGraph-CrewAI architecture that achieves a 96.1% success rate while reducing token consumption by 76.2% and decision latency by $14.5\times $ compared to pure CrewAI implementations. Our comprehensive validation across 51 episodes (17 scenarios $\times 3$ seeds) demonstrates perfect scores in task designation and agent capitalization competencies, outperforming state-of-the-art baselines. This work provides evidence-based guidance for framework selection, highlights fundamental scalability challenges in current systems, and demonstrates that intelligent hybrid designs can overcome individual framework limitations while maintaining cost-efficiency.

Zitieren

APA

Khan, M. S., Khan, M. J., Sharif, M., & Yasmin, S. (2026). Beyond Single-Framework Architectures: A Systematic Evaluation and Hybrid Design for Scalable Multi-Agent Coordination. IEEE Access, 14, 59688-59705. https://doi.org/10.1109/access.2026.3683900

Vancouver

Khan MS, Khan MJ, Sharif M, Yasmin S. Beyond Single-Framework Architectures: A Systematic Evaluation and Hybrid Design for Scalable Multi-Agent Coordination. IEEE Access. 2026;14:59688-59705. doi:10.1109/access.2026.3683900

BibTeX
@article{Khan2026,
  author = {Khan, Muhammad Saleem and Khan, Muhammad Janas and Sharif, Muhammad and Yasmin, Sadaf},
  title = {Beyond Single-Framework Architectures: A Systematic Evaluation and Hybrid Design for Scalable Multi-Agent Coordination},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {59688--59705},
  doi = {10.1109/access.2026.3683900}
}
Kostenlos starten →