EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Comprehensive evaluation of large language models for pediatric pectus excavatum education

Hao Shi, Qiang Tang, Jing Xiong, Hou-Chiang Tseng, Tao-Hsing Chang, Xiaolong Chen, Linhao Peng, Haifa Hong

Frontiers in Public Health · 30.10.2026 · 14

DOI: 10.3389/fpubh.2026.1990753 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Public Health, online erschienen am 30.10.2026. Verfasst von Hao Shi und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of General Surgery, Shanghai Children's Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China. Hauptthemen laut MEDLINE: Large Language Models, Funnel Chest, Patient Education as Topic. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) are increasingly used by patients and caregivers to find health information. However, their performance in providing Simplified Chinese education on pediatric pectus excavatum (PE) remains uncertain. This study comprehensively evaluated the responses generated by five mainstream LLMs.

Methods

Thirty PE-related frequently asked questions (FAQs) spanning five clinical domains were collected and submitted in Simplified Chinese to ChatGPT, Claude, DeepSeek, Qwen, and Kimi. Responses were evaluated by pediatric thoracic surgeons from three institutions for accuracy, completeness, and relevance, using a 6-point Likert scale. Readability was assessed using AlphaReadabilityChinese (ARC) 2.0 and the Chinese Readability Index Explorer (CRIE) 3.0. Understandability, actionability, and overall quality were evaluated with the Chinese Patient Education Materials Assessment Tool for Printable Materials (C-PEMAT-P) and the Global Quality Score (GQS). Friedman tests compared the models, followed by Holm-adjusted Wilcoxon signed-rank tests.

Results

All models achieved mean accuracy, completeness, and relevance scores above 4.9 on the 6-point scale, but the leading model differed by dimension. Claude had the highest mean accuracy (5.39 ± 0.28), completeness (5.22 ± 0.22), and GQS (4.80 ± 0.41), while DeepSeek had the highest relevance (5.42 ± 0.27). Qwen was the least readable: its median CRIE grade was 7.0, compared with 5.5 to 6.0 for the other models. Only 23.3% of its responses were at Grades 1–6, compared with 66.7–86.7% for the others. Overall C-PEMAT-P scores also differed ( p = 0.003), with Kimi showing the highest numerical mean total (12.17 ± 0.75). Qwen combined the highest actionability (2.57 ± 0.50) with the lowest understandability (9.47 ± 0.57) and GQS (3.10 ± 0.71).

Conclusion

Leading LLMs generally provided accurate, complete, and relevant Simplified Chinese answers to common pediatric PE questions, but no model performed best on every dimensions. LLM outputs should therefore be regarded as a supervised educational aid that require expert verification and adaptation to the health literacy of the intended audience, rather than as substitutes for specialist counseling.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsFunnel ChestPatient Education as TopicHumansChinaChildComprehension

Stichwörter der Autor:innen

childrenlarge language modelspatient educationpectus excavatumqualityreadability

Autor:innen und Institutionen

  • Department of General Surgery, Shanghai Children's Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China.
  • Department of General Surgery, Changzhou Children's Hospital of Nantong University, Changzhou, Jiangsu, China.
  • Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, Taipei, Taiwan.
  • Department of Computer Science and Information Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, Taiwan.
  • Department of Cardiothoracic Surgery, Shanghai Children's Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China.
  • School of Artificial Intelligence and Data Science, Shanghai Dianji University, Shanghai, China.

Zitieren

APA

Shi, H., Tang, Q., Xiong, J., Tseng, H. C., Chang, T. H., Chen, X., Peng, L., & Hong, H. (2026). Comprehensive evaluation of large language models for pediatric pectus excavatum education. Frontiers in Public Health, 14. https://doi.org/10.3389/fpubh.2026.1990753

Vancouver

Shi H, Tang Q, Xiong J, Tseng HC, Chang TH, Chen X, et al. Comprehensive evaluation of large language models for pediatric pectus excavatum education. Frontiers in Public Health. 2026;14. doi:10.3389/fpubh.2026.1990753

BibTeX
@article{Shi2026,
  author = {Shi, Hao and Tang, Qiang and Xiong, Jing and Tseng, Hou-Chiang and Chang, Tao-Hsing and Chen, Xiaolong and Peng, Linhao and Hong, Haifa},
  title = {Comprehensive evaluation of large language models for pediatric pectus excavatum education},
  journal = {Frontiers in Public Health},
  year = {2026},
  volume = {14},
  doi = {10.3389/fpubh.2026.1990753}
}
Kostenlos starten →