EUVIMEDSkopri EUVIMED
Randomisierte kontrollierte StudieOpen access

Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial

Hui Ouyang iD, Gan Lin iD, Yiyuan Li iD, Zhixin Yao iD, Yating Li iD, Han Yan iD, Fang Qin iD, Jinghui Yao iD, Yun Chen iD

Journal of Medical Internet Research · 2026-01-19 · 28 · e73326

DOI: 10.2196/73326 ↗

At a glance

Randomisierte kontrollierte Studie in Journal of Medical Internet Research, published online on 2026-01-19. Written by Hui Ouyang and 8 co-authors. Institution: Third Affiliated Hospital of Southern Medical University, Guangzhou, China. Major MEDLINE topics: Developmental Dysplasia of the Hip, Language, Health Education, Patient Education as Topic. Cited 5 times so far (Crossref); 59 references. The full text is freely available in PubMed Central.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Background

Developmental dysplasia of the hip (DDH) is a common pediatric orthopedic disease, and health education is vital to disease management and rehabilitation. The emergence of large language models (LLMs) has provided new opportunities for health education. However, the effectiveness and applicability of LLMs in education with DDH have not been systematically evaluated.

Objective

This study conducted an integrated 2-phase evaluation to assess the quality and educational effectiveness of LLM-generated educational materials.

Methods

This study comprised 2 phases. Based on Bloom’s taxonomy, a 16-item DDH question bank was created through literature analysis and collaboration. Four LLMs (ChatGPT-4 [OpenAI], DeepSeek-V3, Gemini 2.0 Flash [Google], and Copilot [Microsoft Corp]) were questioned using standardized prompts. All responses were independently evaluated by 5 pediatric orthopedic experts using 5-point Likert measures of accuracy, fluency, and richness, the scales of Patient Education Materials Assessment Tool for Printable Materials, and DISCERN. The readability was measured by a formula. The data were examined using Kruskal-Wallis tests, ANOVA, and post hoc comparisons. In phase 2, an assessor-blinded, 2-arm pilot randomized controlled trial was conducted. A total of 127 caregivers were randomized into an LLM-assisted education group or a web search control group. The intervention included structured LLM training, supervised practice, and 2 weeks of reinforcement training. Measured at baseline, postintervention, and 2 weeks following, the outcomes were eHealth literacy (primary), DDH knowledge, health risk perception, perceived usefulness, information self-efficacy, and health information-seeking behavior. Cohen d effect sizes and linear mixed-effects models were used in an intention-to-treat manner.

Results

There were significant differences between the 4 LLMs concerning accuracy, richness, fluency, Patient Education Materials Assessment Tool for Printable Materials Understandability, and DISCERN (P

Subjects

MeSH terms (MEDLINE)

Developmental Dysplasia of the HipLanguageHealth EducationPatient Education as TopicHumansPilot ProjectsFemaleMaleLarge Language Models

Author keywords

content generationdevelopmental dysplasia of the hiphealth educationlarge language modelsmobile phone

Authors and institutions

  • Third Affiliated Hospital of Southern Medical University, Guangzhou, China.
  • School of Nursing, Southern Medical University, Guangzhou, China.
  • Department of Pediatric Orthopedics, Center for Orthopaedic Surgery, Third Affiliated Hospital of Southern Medical University, Guangzhou, China.

ORCID profiles: Hui Ouyang, Gan Lin, Yiyuan Li, Zhixin Yao, Yating Li, Han Yan, Fang Qin, Jinghui Yao, Yun Chen

Cite

APA

Ouyang, H., Lin, G., Li, Y., Yao, Z., Li, Y., Yan, H., Qin, F., Yao, J., & Chen, Y. (2026). Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial. Journal of Medical Internet Research, 28, e73326. https://doi.org/10.2196/73326

Vancouver

Ouyang H, Lin G, Li Y, Yao Z, Li Y, Yan H, et al. Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial. Journal of Medical Internet Research. 2026;28:e73326. doi:10.2196/73326

BibTeX
@article{Ouyang2026,
  author = {Ouyang, Hui and Lin, Gan and Li, Yiyuan and Yao, Zhixin and Li, Yating and Yan, Han and Qin, Fang and Yao, Jinghui and Chen, Yun},
  title = {Evaluating and Validating Large Language Models for Health Education on Developmental Dysplasia of the Hip: 2-Phase Study With Expert Ratings and a Pilot Randomized Controlled Trial},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e73326},
  doi = {10.2196/73326}
}
Start for free →