Orchestrator multi-agent clinical decision support system for secondary headache diagnosis in primary care
Journal of the American Medical Informatics Association · 01.10.2026 · 33 (10) · 1910-1920
DOI: 10.1093/jamia/ocag111 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 27.06.2026. Verfasst von Xizhi Wu und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States. Hauptthemen laut MEDLINE: Decision Support Systems, Clinical, Large Language Models, Headache. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NCATS NIH HHS, NIH HHS, NLM NIH HHS.
Abstract (Originalsprache)
Objective
Secondary headaches require urgent recognition due to potentially devastating consequences if untreated. Despite established clinical “red flag” criteria, identifying patients needing immediate evaluation remains challenging in primary care. This study developed and evaluated a large language model (LLM)–based multi-agent clinical decision support system for interpretable secondary headache diagnosis. Materials and Methods We first established 7 clinically relevant secondary headache red flag domains through manual review and synthesis of clinical guidelines. Based on these domains, we designed an LLM-based system using an orchestrator–specialist multi-agent architecture that decomposes diagnostic reasoning into 7 guideline-aligned agents corresponding to key red flag features. Each agent generates structured, evidence-grounded reasoning, coordinated by a central orchestrator. The system was evaluated on 90 expert-validated secondary headache cases and compared with a single-LLM baseline under 2 prompting strategies: question-based prompting (QPrompt) and guideline-based prompting (GPrompt). Five open-source LLMs (Qwen-8b, Qwen-14b, Qwen-30b, GPT-OSS-20b, and Llama-3.1-8b) were tested.
Results
The orchestrated multi-agent system with GPrompt achieved the highest red flag classification performance across models, measured by F1 score. Performance gains were consistent and more pronounced in smaller LLMs, suggesting that structured reasoning improves efficiency and accuracy beyond prompt engineering alone. The framework also produced transparent and guideline-aligned intermediate reasoning.
Discussion
Decomposing clinical reasoning into specialized agents enhances interpretability and diagnostic reliability compared with monolithic LLM approaches. Multi-agent orchestration provides a clinically aligned framework for explainable decision support.
Conclusion
An orchestrator–specialist multi-agent LLM framework improves secondary headache diagnosis accuracy and transparency, supporting the development of explainable AI systems for time-constrained clinical decision-making in primary care.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, United States.
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, United States.
- Clinical Informatics Center, UT Southwestern Medical Center, Dallas, TX, United States.
- Peter O'Donnell Jr. Brain Institute, UT Southwestern Medical Center, Dallas, TX, United States.
- Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, United States.
- Population Health Sciences, Weill Cornell Medicine, New York, NY, United States.
ORCID-Profile: Justin F Rousseau, Shyam Visweswaran, Yifan Peng, Yanshan Wang
Förderung
- NCATS NIH HHS
- NIH HHS
- NLM NIH HHS
- National Institutes of Health
Zitieren
APA
Wu, X., Zhang, H., Garduno-Rapp, N. E., Rousseau, J. F., Thakkallapally, M., Ji, Y., Visweswaran, S., Peng, Y., & Wang, Y. (2026). Orchestrator multi-agent clinical decision support system for secondary headache diagnosis in primary care. Journal of the American Medical Informatics Association, 33(10), 1910-1920. https://doi.org/10.1093/jamia/ocag111
Vancouver
Wu X, Zhang H, Garduno-Rapp NE, Rousseau JF, Thakkallapally M, Ji Y, et al. Orchestrator multi-agent clinical decision support system for secondary headache diagnosis in primary care. Journal of the American Medical Informatics Association. 2026;33(10):1910-1920. doi:10.1093/jamia/ocag111
BibTeX
@article{Wu2026,
author = {Wu, Xizhi and Zhang, Hang and Garduno-Rapp, Nelly Estefanie and Rousseau, Justin F and Thakkallapally, Mounika and Ji, Yuelyu and Visweswaran, Shyam and Peng, Yifan and Wang, Yanshan},
title = {Orchestrator multi-agent clinical decision support system for secondary headache diagnosis in primary care},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
volume = {33},
number = {10},
pages = {1910--1920},
doi = {10.1093/jamia/ocag111}
}
EUVIMED