Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial
Journal of Medical Internet Research · 2026-01-19 · 28 · e73326
DOI: 10.2196/73326 ↗At a glance
Randomisierte kontrollierte Studie in Journal of Medical Internet Research, published online on 2026-01-19. Written by Hui Ouyang and 8 co-authors. Institution: Third Affiliated Hospital of Southern Medical University, Guangzhou, China. Major MEDLINE topics: Developmental Dysplasia of the Hip, Language, Health Education, Patient Education as Topic. Cited 5 times so far (Crossref); 59 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Developmental dysplasia of the hip (DDH) is a common pediatric orthopedic disease, and health education is vital to disease management and rehabilitation. The emergence of large language models (LLMs) has provided new opportunities for health education. However, the effectiveness and applicability of LLMs in education with DDH have not been systematically evaluated.
Objective
This study conducted an integrated 2-phase evaluation to assess the quality and educational effectiveness of LLM-generated educational materials.
Methods
This study comprised 2 phases. Based on Bloom’s taxonomy, a 16-item DDH question bank was created through literature analysis and collaboration. Four LLMs (ChatGPT-4 [OpenAI], DeepSeek-V3, Gemini 2.0 Flash [Google], and Copilot [Microsoft Corp]) were questioned using standardized prompts. All responses were independently evaluated by 5 pediatric orthopedic experts using 5-point Likert measures of accuracy, fluency, and richness, the scales of Patient Education Materials Assessment Tool for Printable Materials, and DISCERN. The readability was measured by a formula. The data were examined using Kruskal-Wallis tests, ANOVA, and post hoc comparisons. In phase 2, an assessor-blinded, 2-arm pilot randomized controlled trial was conducted. A total of 127 caregivers were randomized into an LLM-assisted education group or a web search control group. The intervention included structured LLM training, supervised practice, and 2 weeks of reinforcement training. Measured at baseline, postintervention, and 2 weeks following, the outcomes were eHealth literacy (primary), DDH knowledge, health risk perception, perceived usefulness, information self-efficacy, and health information-seeking behavior. Cohen d effect sizes and linear mixed-effects models were used in an intention-to-treat manner.
Results
There were significant differences between the 4 LLMs concerning accuracy, richness, fluency, Patient Education Materials Assessment Tool for Printable Materials Understandability, and DISCERN (P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Third Affiliated Hospital of Southern Medical University, Guangzhou, China.
- School of Nursing, Southern Medical University, Guangzhou, China.
- Department of Pediatric Orthopedics, Center for Orthopaedic Surgery, Third Affiliated Hospital of Southern Medical University, Guangzhou, China.
ORCID profiles: Hui Ouyang, Gan Lin, Yiyuan Li, Zhixin Yao, Yating Li, Han Yan, Fang Qin, Jinghui Yao, Yun Chen
Cite
APA
Ouyang, H., Lin, G., Li, Y., Yao, Z., Li, Y., Yan, H., Qin, F., Yao, J., & Chen, Y. (2026). Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial. Journal of Medical Internet Research, 28, e73326. https://doi.org/10.2196/73326
Vancouver
Ouyang H, Lin G, Li Y, Yao Z, Li Y, Yan H, et al. Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial. Journal of Medical Internet Research. 2026;28:e73326. doi:10.2196/73326
BibTeX
@article{Ouyang2026,
author = {Ouyang, Hui and Lin, Gan and Li, Yiyuan and Yao, Zhixin and Li, Yating and Yan, Han and Qin, Fang and Yao, Jinghui and Chen, Yun},
title = {Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e73326},
doi = {10.2196/73326}
}
EUVIMED