Comparative benchmark of four selected large language models for generating patient education content in pediatric neurology: a prospective study
Frontiers in Pediatrics · 16.11.2026 · 14
DOI: 10.3389/fped.2026.1901603 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Pediatrics, online erschienen am 16.11.2026. Verfasst von Osama Muthaffar und 6 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Abdulaziz University.
Abstract (Originalsprache)
Background
Effective patient education is essential in pediatric neurology but challenged by time constraints and variable health literacy. Large language models (LLMs) offer a potential solution, yet their accuracy and safety in this specialized field remain incompletely evaluated.
Methods
In a prospective, blinded benchmarking study, LLM-generated responses to ten standardized pediatric neurology prompts were evaluated. Responses were assessed for factual accuracy, safety, and perceived usability; the study did not assess real-world clinical decision support or caregiver comprehension. A blinded panel of seven consultant pediatric neurology evaluators assessed the outputs using an expert-developed rubric measuring factual accuracy, safety, clinical appropriateness, and usability. The unit of analysis was the scenario-model pair ( n = 40), with median scores across raters used for primary comparisons.
Results
Linear mixed-effects models revealed significant performance variation across models (F(3,27) = 8.76, p < 0.001). DeepSeek achieved the highest overall composite score (4.01 ± 0.61) and zero safety flags. ChatGPT scored moderately (3.72 ± 0.75) but accounted for most safety issues (5/8), including a contraindicated medication recommendation. Gemini and Perplexity scored lower (3.45 ± 0.77 and 3.38 ± 0.71). The overall composite difference between DeepSeek and ChatGPT was not statistically significant ( p = 0.087). DeepSeek had the highest System Usability Scale score (74.2/100). Presence of any consensus safety flag drastically reduced the odds of expert approval for caregiver-facing use after clinician review (OR: 0.08, p < 0.001).
Conclusions
DeepSeek produced no consensus safety flags in this benchmark and had the highest expert-rated usability score, but its overall composite score was not statistically different from ChatGPT. These findings are limited to the scenarios tested and do not establish broad safety, clinical effectiveness, or caregiver benefit. No caregivers were directly studied. No model is ready for autonomous use, and clinician verification remains mandatory. This benchmark provides a methodology for specialty-specific evaluation.
Förderung
- King Abdulaziz University
Zitieren
APA
Muthaffar, O., Alyazidi, A., Awadalla, M., Alanezi, A. A., Alsehemi, M., Hothan, K., & Naseer, M. I. (2026). Comparative benchmark of four selected large language models for generating patient education content in pediatric neurology: a prospective study. Frontiers in Pediatrics, 14. https://doi.org/10.3389/fped.2026.1901603
Vancouver
Muthaffar O, Alyazidi A, Awadalla M, Alanezi AA, Alsehemi M, Hothan K, et al. Comparative benchmark of four selected large language models for generating patient education content in pediatric neurology: a prospective study. Frontiers in Pediatrics. 2026;14. doi:10.3389/fped.2026.1901603
BibTeX
@article{Muthaffar2026,
author = {Muthaffar, Osama and Alyazidi, Anas and Awadalla, Maha and Alanezi, Ahmad A. and Alsehemi, Matar and Hothan, Kholoud and Naseer, Muhammad Imran},
title = {Comparative benchmark of four selected large language models for generating patient education content in pediatric neurology: a prospective study},
journal = {Frontiers in Pediatrics},
year = {2026},
volume = {14},
doi = {10.3389/fped.2026.1901603}
}
EUVIMED