EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Comparative benchmark of four selected large language models for generating patient education content in pediatric neurology: a prospective study

Osama Muthaffar, Anas Alyazidi, Maha Awadalla, Ahmad A. Alanezi, Matar Alsehemi, Kholoud Hothan, Muhammad Imran Naseer

Frontiers in Pediatrics · 16.11.2026 · 14

DOI: 10.3389/fped.2026.1901603 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Pediatrics, online erschienen am 16.11.2026. Verfasst von Osama Muthaffar und 6 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Abdulaziz University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Effective patient education is essential in pediatric neurology but challenged by time constraints and variable health literacy. Large language models (LLMs) offer a potential solution, yet their accuracy and safety in this specialized field remain incompletely evaluated.

Methods

In a prospective, blinded benchmarking study, LLM-generated responses to ten standardized pediatric neurology prompts were evaluated. Responses were assessed for factual accuracy, safety, and perceived usability; the study did not assess real-world clinical decision support or caregiver comprehension. A blinded panel of seven consultant pediatric neurology evaluators assessed the outputs using an expert-developed rubric measuring factual accuracy, safety, clinical appropriateness, and usability. The unit of analysis was the scenario-model pair ( n = 40), with median scores across raters used for primary comparisons.

Results

Linear mixed-effects models revealed significant performance variation across models (F(3,27) = 8.76, p < 0.001). DeepSeek achieved the highest overall composite score (4.01 ± 0.61) and zero safety flags. ChatGPT scored moderately (3.72 ± 0.75) but accounted for most safety issues (5/8), including a contraindicated medication recommendation. Gemini and Perplexity scored lower (3.45 ± 0.77 and 3.38 ± 0.71). The overall composite difference between DeepSeek and ChatGPT was not statistically significant ( p = 0.087). DeepSeek had the highest System Usability Scale score (74.2/100). Presence of any consensus safety flag drastically reduced the odds of expert approval for caregiver-facing use after clinician review (OR: 0.08, p < 0.001).

Conclusions

DeepSeek produced no consensus safety flags in this benchmark and had the highest expert-rated usability score, but its overall composite score was not statistically different from ChatGPT. These findings are limited to the scenarios tested and do not establish broad safety, clinical effectiveness, or caregiver benefit. No caregivers were directly studied. No model is ready for autonomous use, and clinician verification remains mandatory. This benchmark provides a methodology for specialty-specific evaluation.

Förderung

  • King Abdulaziz University

Zitieren

APA

Muthaffar, O., Alyazidi, A., Awadalla, M., Alanezi, A. A., Alsehemi, M., Hothan, K., & Naseer, M. I. (2026). Comparative benchmark of four selected large language models for generating patient education content in pediatric neurology: a prospective study. Frontiers in Pediatrics, 14. https://doi.org/10.3389/fped.2026.1901603

Vancouver

Muthaffar O, Alyazidi A, Awadalla M, Alanezi AA, Alsehemi M, Hothan K, et al. Comparative benchmark of four selected large language models for generating patient education content in pediatric neurology: a prospective study. Frontiers in Pediatrics. 2026;14. doi:10.3389/fped.2026.1901603

BibTeX
@article{Muthaffar2026,
  author = {Muthaffar, Osama and Alyazidi, Anas and Awadalla, Maha and Alanezi, Ahmad A. and Alsehemi, Matar and Hothan, Kholoud and Naseer, Muhammad Imran},
  title = {Comparative benchmark of four selected large language models for generating patient education content in pediatric neurology: a prospective study},
  journal = {Frontiers in Pediatrics},
  year = {2026},
  volume = {14},
  doi = {10.3389/fped.2026.1901603}
}
Kostenlos starten →