EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Performance of Large Language Models in Neurology Multiple‐Choice Questions

Gholamreza Habibi, Omid Kohandel Gargari, Mostafa Hosseini, Kasra Afchangi

Acta Neurologica Scandinavica · 2026-01 · 2026 (1)

DOI: 10.1155/ane/5623086 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Acta Neurologica Scandinavica, online erschienen am 01.06.2026. Verfasst von Gholamreza Habibi und 3 weiteren Autor:innen. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Navigating neurological disorders is complex due to overlapping symptoms and diverse diagnostic requirements. Large language models (LLMs) offer potential support for clinicians by processing vast amounts of textual data. This study evaluates the performance of four advanced LLMs—GPT‐4, GPT‐3.5, Clinical Camel, and MedALPACA—in answering neurology‐based multiple‐choice questions (MCQs).

Methods

The study utilized 170 MCQs from the Comprehensive Review in Clinical Neurology book. Questions were divided into the body and answer choices, stored separately in an Excel spreadsheet. The models were prompted to select the correct answers. Generative Pretrained Transformer (GPT) models were accessed via OpenAI′s API, whereas Clinical Camel and MedALPACA were downloaded from Hugging Face. Accuracy was calculated by the number of correct answers over total questions, with subgroup analysis based on subject headings.

Results

GPT‐4 achieved the highest accuracy at 84.7%, significantly outperforming GPT‐3.5 (58.8%), Clinical Camel (52.9%), and MedALPACA (40%). GPT‐4′s performance was significantly better ( p < 0.001). The difference between GPT‐3.5 and Clinical Camel was not significant ( p = 0.27), but both outperformed MedALPACA. Response similarity was highest between GPT‐4 and GPT‐3.5 (64.1%) and lowest between GPT‐4 and MedALPACA (41.2%). In subgroup analysis, GPT‐4 was superior across all topics, achieving full scores in six topics and its lowest in vascular neurology (40%). GPT‐3.5 performed best in eight topics, Clinical Camel in three topics, and MedALPACA was the weakest in all but two topics.

Conclusion

GPT‐4 demonstrated the highest accuracy in answering neurology MCQs, outperforming GPT‐3.5, Clinical Camel, and MedALPACA. Clinical Camel′s comparable performance to GPT‐3.5 highlights the potential of specialized medical models. External evaluation datasets remain essential to avoid data leakage and ensure fair benchmarking. Further research is needed to expand topic coverage, assess reasoning processes, and include human comparison to support safe and effective clinical integration of medical LLMs.

Zitieren

APA

Habibi, G., Gargari, O. K., Hosseini, M., & Afchangi, K. (2026). Performance of Large Language Models in Neurology Multiple‐Choice Questions. Acta Neurologica Scandinavica, 2026(1). https://doi.org/10.1155/ane/5623086

Vancouver

Habibi G, Gargari OK, Hosseini M, Afchangi K. Performance of Large Language Models in Neurology Multiple‐Choice Questions. Acta Neurologica Scandinavica. 2026;2026(1). doi:10.1155/ane/5623086

BibTeX
@article{Habibi2026,
  author = {Habibi, Gholamreza and Gargari, Omid Kohandel and Hosseini, Mostafa and Afchangi, Kasra},
  title = {Performance of Large Language Models in Neurology Multiple‐Choice Questions},
  journal = {Acta Neurologica Scandinavica},
  year = {2026},
  volume = {2026},
  number = {1},
  doi = {10.1155/ane/5623086}
}
Kostenlos starten →