EUVIMEDEUVIMED entdecken
Vergleichsstudie

AI for clinical neurology: Comparing five large language models in neurological diagnosis and lesion localization

Francesco Brigo, Arian Zaboli, Elisabetta Maida, Luigi Lavorgna

Neurological Sciences · 2026

DOI: 10.1007/s10072-026-08847-2 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Neurological Sciences, 2026. Verfasst von Francesco Brigo und 3 weiteren Autor:innen. Beteiligte Einrichtung: Innovation, Research and Teaching Service, Teaching Hospital of the Paracelsus Medical Private University, South Tyrolean Health Authority (SABES-ASD…. Hauptthemen laut MEDLINE: Neurology, Nervous System Diseases. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 12 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: Large language models (LLMs) are proposed as decision-support tools in medicine, but their ability to reason through neurological cases—requiring anatomoclinical correlation—remains uncertain. METHODS: Five LLMs (Copilot GPT-5, DeepSeek-V3.1, ChatGPT 3.5, ChatGPT 5 Auto, Gemini 2.5) were tested on 86 neurological vignettes. Outputs were graded as Correct, Partial, or Incorrect for diagnosis and lesion site. Accuracy was analyzed using binary (correct vs. not correct), weighted (partial = 0.5), and ordinal (0–1–2 scale) methods. Global comparisons employed Cochran’s Q or Friedman tests; pairwise differences used McNemar or Wilcoxon tests with Holm correction. Interrater reliability was assessed with Fleiss’ κ, Cohen’s κ, and raw percent agreement. RESULTS: For diagnosis, ChatGPT 5 Auto, Gemini 2.5, Copilot GPT-5, and DeepSeek-V3.1 each achieved > 90% accuracy, all significantly outperforming ChatGPT 3.5. For lesion localization, ChatGPT 5 Auto achieved the highest diagnostic accuracy, followed closely by Copilot GPT-5 and Gemini, while ChatGPT-3.5 lagged behind and had the highest rate of incorrect responses (13.9%). Weighted and ordinal analyses confirmed the same rank order, clustering the four advanced models together. Agreement was limited for diagnosis (Fleiss’ κ = 0.22) and for localization (Fleiss’ κ = 0.20). CONCLUSION: Advanced LLMs demonstrate high and comparable performance in neurology, while ChatGPT 3.5 is significantly less accurate. Errors were heterogeneous across models, suggesting potential ensemble benefits but reinforcing the need for human oversight, particularly in the irreplaceable human tasks of history‑taking, examination, and contextual clinical judgment.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

NeurologyNervous System DiseasesHumansLarge Language ModelsGenerative Artificial IntelligenceReproducibility of Results

Stichwörter der Autor:innen

AccuracyArtificial IntelligenceDiagnosisLarge Language ModelsNeurology

Autor:innen und Institutionen

  • Innovation, Research and Teaching Service, Teaching Hospital of the Paracelsus Medical Private University, South Tyrolean Health Authority (SABES-ASDAA), Azienda Sanitaria Dell'Alto Adige; Via A. Volta, 13, Bolzano-Boze…
  • Health Professions Management, South Tyrolean Health Authority (SABES-ASDAA), Bolzano-Bozen, Italy.
  • Department of Advanced Medical and Surgical Sciences, University of Campania Luigi Vanvitelli, Naples, Italy.

Zitieren

APA

Brigo, F., Zaboli, A., Maida, E., & Lavorgna, L. (2026). AI for clinical neurology: Comparing five large language models in neurological diagnosis and lesion localization. Neurological Sciences. https://doi.org/10.1007/s10072-026-08847-2

Vancouver

Brigo F, Zaboli A, Maida E, Lavorgna L. AI for clinical neurology: Comparing five large language models in neurological diagnosis and lesion localization. Neurological Sciences. 2026. doi:10.1007/s10072-026-08847-2

BibTeX
@article{Brigo2026,
  author = {Brigo, Francesco and Zaboli, Arian and Maida, Elisabetta and Lavorgna, Luigi},
  title = {AI for clinical neurology: Comparing five large language models in neurological diagnosis and lesion localization},
  journal = {Neurological Sciences},
  year = {2026},
  doi = {10.1007/s10072-026-08847-2}
}
Kostenlos starten →