Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial
Journal of Medical Internet Research · 19.01.2026 · 28 · e73326
DOI: 10.2196/73326 ↗Kurz gefasst
Randomisierte kontrollierte Studie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 19.01.2026. Verfasst von Hui Ouyang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Third Affiliated Hospital of Southern Medical University, Guangzhou, China. Hauptthemen laut MEDLINE: Developmental Dysplasia of the Hip, Language, Health Education, Patient Education as Topic. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Developmental dysplasia of the hip (DDH) is a common pediatric orthopedic disease, and health education is vital to disease management and rehabilitation. The emergence of large language models (LLMs) has provided new opportunities for health education. However, the effectiveness and applicability of LLMs in education with DDH have not been systematically evaluated.
Objective
This study conducted an integrated 2-phase evaluation to assess the quality and educational effectiveness of LLM-generated educational materials.
Methods
This study comprised 2 phases. Based on Bloom’s taxonomy, a 16-item DDH question bank was created through literature analysis and collaboration. Four LLMs (ChatGPT-4 [OpenAI], DeepSeek-V3, Gemini 2.0 Flash [Google], and Copilot [Microsoft Corp]) were questioned using standardized prompts. All responses were independently evaluated by 5 pediatric orthopedic experts using 5-point Likert measures of accuracy, fluency, and richness, the scales of Patient Education Materials Assessment Tool for Printable Materials, and DISCERN. The readability was measured by a formula. The data were examined using Kruskal-Wallis tests, ANOVA, and post hoc comparisons. In phase 2, an assessor-blinded, 2-arm pilot randomized controlled trial was conducted. A total of 127 caregivers were randomized into an LLM-assisted education group or a web search control group. The intervention included structured LLM training, supervised practice, and 2 weeks of reinforcement training. Measured at baseline, postintervention, and 2 weeks following, the outcomes were eHealth literacy (primary), DDH knowledge, health risk perception, perceived usefulness, information self-efficacy, and health information-seeking behavior. Cohen d effect sizes and linear mixed-effects models were used in an intention-to-treat manner.
Results
There were significant differences between the 4 LLMs concerning accuracy, richness, fluency, Patient Education Materials Assessment Tool for Printable Materials Understandability, and DISCERN (P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Third Affiliated Hospital of Southern Medical University, Guangzhou, China.
- School of Nursing, Southern Medical University, Guangzhou, China.
- Department of Pediatric Orthopedics, Center for Orthopaedic Surgery, Third Affiliated Hospital of Southern Medical University, Guangzhou, China.
ORCID-Profile: Hui Ouyang, Gan Lin, Yiyuan Li, Zhixin Yao, Yating Li, Han Yan, Fang Qin, Jinghui Yao, Yun Chen
Zitieren
APA
Ouyang, H., Lin, G., Li, Y., Yao, Z., Li, Y., Yan, H., Qin, F., Yao, J., & Chen, Y. (2026). Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial. Journal of Medical Internet Research, 28, e73326. https://doi.org/10.2196/73326
Vancouver
Ouyang H, Lin G, Li Y, Yao Z, Li Y, Yan H, et al. Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial. Journal of Medical Internet Research. 2026;28:e73326. doi:10.2196/73326
BibTeX
@article{Ouyang2026,
author = {Ouyang, Hui and Lin, Gan and Li, Yiyuan and Yao, Zhixin and Li, Yating and Yan, Han and Qin, Fang and Yao, Jinghui and Chen, Yun},
title = {Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e73326},
doi = {10.2196/73326}
}
EUVIMED