AI for clinical neurology: Comparing five large language models in neurological diagnosis and lesion localization
Neurological Sciences · 2026
DOI: 10.1007/s10072-026-08847-2 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Neurological Sciences, 2026. Verfasst von Francesco Brigo und 3 weiteren Autor:innen. Beteiligte Einrichtung: Innovation, Research and Teaching Service, Teaching Hospital of the Paracelsus Medical Private University, South Tyrolean Health Authority (SABES-ASD…. Hauptthemen laut MEDLINE: Neurology, Nervous System Diseases. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 12 Literaturangaben.
Abstract (Originalsprache)
BACKGROUND: Large language models (LLMs) are proposed as decision-support tools in medicine, but their ability to reason through neurological cases—requiring anatomoclinical correlation—remains uncertain. METHODS: Five LLMs (Copilot GPT-5, DeepSeek-V3.1, ChatGPT 3.5, ChatGPT 5 Auto, Gemini 2.5) were tested on 86 neurological vignettes. Outputs were graded as Correct, Partial, or Incorrect for diagnosis and lesion site. Accuracy was analyzed using binary (correct vs. not correct), weighted (partial = 0.5), and ordinal (0–1–2 scale) methods. Global comparisons employed Cochran’s Q or Friedman tests; pairwise differences used McNemar or Wilcoxon tests with Holm correction. Interrater reliability was assessed with Fleiss’ κ, Cohen’s κ, and raw percent agreement. RESULTS: For diagnosis, ChatGPT 5 Auto, Gemini 2.5, Copilot GPT-5, and DeepSeek-V3.1 each achieved > 90% accuracy, all significantly outperforming ChatGPT 3.5. For lesion localization, ChatGPT 5 Auto achieved the highest diagnostic accuracy, followed closely by Copilot GPT-5 and Gemini, while ChatGPT-3.5 lagged behind and had the highest rate of incorrect responses (13.9%). Weighted and ordinal analyses confirmed the same rank order, clustering the four advanced models together. Agreement was limited for diagnosis (Fleiss’ κ = 0.22) and for localization (Fleiss’ κ = 0.20). CONCLUSION: Advanced LLMs demonstrate high and comparable performance in neurology, while ChatGPT 3.5 is significantly less accurate. Errors were heterogeneous across models, suggesting potential ensemble benefits but reinforcing the need for human oversight, particularly in the irreplaceable human tasks of history‑taking, examination, and contextual clinical judgment.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Innovation, Research and Teaching Service, Teaching Hospital of the Paracelsus Medical Private University, South Tyrolean Health Authority (SABES-ASDAA), Azienda Sanitaria Dell'Alto Adige; Via A. Volta, 13, Bolzano-Boze…
- Health Professions Management, South Tyrolean Health Authority (SABES-ASDAA), Bolzano-Bozen, Italy.
- Department of Advanced Medical and Surgical Sciences, University of Campania Luigi Vanvitelli, Naples, Italy.
Zitieren
APA
Brigo, F., Zaboli, A., Maida, E., & Lavorgna, L. (2026). AI for clinical neurology: Comparing five large language models in neurological diagnosis and lesion localization. Neurological Sciences. https://doi.org/10.1007/s10072-026-08847-2
Vancouver
Brigo F, Zaboli A, Maida E, Lavorgna L. AI for clinical neurology: Comparing five large language models in neurological diagnosis and lesion localization. Neurological Sciences. 2026. doi:10.1007/s10072-026-08847-2
BibTeX
@article{Brigo2026,
author = {Brigo, Francesco and Zaboli, Arian and Maida, Elisabetta and Lavorgna, Luigi},
title = {AI for clinical neurology: Comparing five large language models in neurological diagnosis and lesion localization},
journal = {Neurological Sciences},
year = {2026},
doi = {10.1007/s10072-026-08847-2}
}
EUVIMED