Comprehensive evaluation of large language models for pediatric pectus excavatum education
Frontiers in Public Health · 30.10.2026 · 14
DOI: 10.3389/fpubh.2026.1990753 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Public Health, online erschienen am 30.10.2026. Verfasst von Hao Shi und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of General Surgery, Shanghai Children's Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China. Hauptthemen laut MEDLINE: Large Language Models, Funnel Chest, Patient Education as Topic. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Large language models (LLMs) are increasingly used by patients and caregivers to find health information. However, their performance in providing Simplified Chinese education on pediatric pectus excavatum (PE) remains uncertain. This study comprehensively evaluated the responses generated by five mainstream LLMs.
Methods
Thirty PE-related frequently asked questions (FAQs) spanning five clinical domains were collected and submitted in Simplified Chinese to ChatGPT, Claude, DeepSeek, Qwen, and Kimi. Responses were evaluated by pediatric thoracic surgeons from three institutions for accuracy, completeness, and relevance, using a 6-point Likert scale. Readability was assessed using AlphaReadabilityChinese (ARC) 2.0 and the Chinese Readability Index Explorer (CRIE) 3.0. Understandability, actionability, and overall quality were evaluated with the Chinese Patient Education Materials Assessment Tool for Printable Materials (C-PEMAT-P) and the Global Quality Score (GQS). Friedman tests compared the models, followed by Holm-adjusted Wilcoxon signed-rank tests.
Results
All models achieved mean accuracy, completeness, and relevance scores above 4.9 on the 6-point scale, but the leading model differed by dimension. Claude had the highest mean accuracy (5.39 ± 0.28), completeness (5.22 ± 0.22), and GQS (4.80 ± 0.41), while DeepSeek had the highest relevance (5.42 ± 0.27). Qwen was the least readable: its median CRIE grade was 7.0, compared with 5.5 to 6.0 for the other models. Only 23.3% of its responses were at Grades 1–6, compared with 66.7–86.7% for the others. Overall C-PEMAT-P scores also differed ( p = 0.003), with Kimi showing the highest numerical mean total (12.17 ± 0.75). Qwen combined the highest actionability (2.57 ± 0.50) with the lowest understandability (9.47 ± 0.57) and GQS (3.10 ± 0.71).
Conclusion
Leading LLMs generally provided accurate, complete, and relevant Simplified Chinese answers to common pediatric PE questions, but no model performed best on every dimensions. LLM outputs should therefore be regarded as a supervised educational aid that require expert verification and adaptation to the health literacy of the intended audience, rather than as substitutes for specialist counseling.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of General Surgery, Shanghai Children's Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China.
- Department of General Surgery, Changzhou Children's Hospital of Nantong University, Changzhou, Jiangsu, China.
- Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, Taipei, Taiwan.
- Department of Computer Science and Information Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, Taiwan.
- Department of Cardiothoracic Surgery, Shanghai Children's Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China.
- School of Artificial Intelligence and Data Science, Shanghai Dianji University, Shanghai, China.
Zitieren
APA
Shi, H., Tang, Q., Xiong, J., Tseng, H. C., Chang, T. H., Chen, X., Peng, L., & Hong, H. (2026). Comprehensive evaluation of large language models for pediatric pectus excavatum education. Frontiers in Public Health, 14. https://doi.org/10.3389/fpubh.2026.1990753
Vancouver
Shi H, Tang Q, Xiong J, Tseng HC, Chang TH, Chen X, et al. Comprehensive evaluation of large language models for pediatric pectus excavatum education. Frontiers in Public Health. 2026;14. doi:10.3389/fpubh.2026.1990753
BibTeX
@article{Shi2026,
author = {Shi, Hao and Tang, Qiang and Xiong, Jing and Tseng, Hou-Chiang and Chang, Tao-Hsing and Chen, Xiaolong and Peng, Linhao and Hong, Haifa},
title = {Comprehensive evaluation of large language models for pediatric pectus excavatum education},
journal = {Frontiers in Public Health},
year = {2026},
volume = {14},
doi = {10.3389/fpubh.2026.1990753}
}
EUVIMED