EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Evaluating Large Language Models for Preoperative Patient Education in Superior Capsular Reconstruction: Comparative Study of Claude, GPT, and Gemini

Yukang Liu iD, Hua Li iD, Jianfeng Ouyang iD, Zhaowen Xue iD, Min Wang iD, Hebei He iD, Bin Song iD, Xiaofei Zheng iD, Wenyi Gan iD

JMIR Perioperative Medicine · 12.06.2025 · 8 · e70047-e70047

DOI: 10.2196/70047 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Perioperative Medicine, online erschienen am 12.06.2025. Verfasst von Yukang Liu und 8 weiteren Autor:innen. Beteiligte Einrichtung: The Second School of Clinical Medicine, Southern Medical University, Guangzhou, China. Hauptthemen laut MEDLINE: Patient Education as Topic, Plastic Surgery Procedures, Preoperative Care, Natural Language Processing. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) are revolutionizing natural language processing, increasingly applied in clinical settings to enhance preoperative patient education.

Objective

This study aimed to evaluate the effectiveness and applicability of various LLMs in preoperative patient education by analyzing their responses to superior capsular reconstruction (SCR)–related inquiries.

Methods

In total, 10 sports medicine clinical experts formulated 11 SCR issues and developed preoperative patient education strategies during a webinar, inputting 12 text commands into Claude-3-Opus (Anthropic), GPT-4-Turbo (OpenAI), and Gemini-1.5-Pro (Google DeepMind). A total of 3 experts assessed the language models’ responses for correctness, completeness, logic, potential harm, and overall satisfaction, while preoperative education documents were evaluated using DISCERN questionnaire and Patient Education Materials Assessment Tool instruments, and reviewed by 5 postoperative patients for readability and educational value; readability of all responses was also analyzed using the cntext package and py-readability-metrics.

Results

Between July 1 and August 17, 2024, sports medicine experts and patients evaluated 33 responses and 3 preoperative patient education documents generated by 3 language models regarding SCR surgery. For the 11 query responses, clinicians rated Gemini significantly higher than Claude in all categories ( P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Patient Education as TopicPlastic Surgery ProceduresPreoperative CareNatural Language ProcessingHumansMaleFemaleSurveys and QuestionnairesMiddle AgedAdultLarge Language Models

Stichwörter der Autor:innen

informed consent processlarge language modelsmassive rotator cuff tearpreoperative patient educationsuperior capsular reconstruction

Autor:innen und Institutionen

  • The Second School of Clinical Medicine, Southern Medical University, Guangzhou, China.
  • Department of Orthopedics, Beijing Jishuitan Hospital, Beijing, China.
  • Zhuhai People's Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University), 79 Kangning Road, Xiangzhou District, Zhuhai, Guangdong, 519000, China, 86 1307…
  • Department of Sports Medicine, The First Affiliated Hospital of Jinan University, Guangzhou, China.
  • Department of Orthopaedics, Guangzhou Red Cross Hospital of Jinan University, Guangzhou, China.
  • Department of Joint Surgery and Sports Medicine, The Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China.

ORCID-Profile: Yukang Liu, Hua Li, Jianfeng Ouyang, Zhaowen Xue, Min Wang, Hebei He, Bin Song, Xiaofei Zheng, Wenyi Gan

Zitieren

APA

Liu, Y., Li, H., Ouyang, J., Xue, Z., Wang, M., He, H., Song, B., Zheng, X., & Gan, W. (2025). Evaluating Large Language Models for Preoperative Patient Education in Superior Capsular Reconstruction: Comparative Study of Claude, GPT, and Gemini. JMIR Perioperative Medicine, 8, e70047-e70047. https://doi.org/10.2196/70047

Vancouver

Liu Y, Li H, Ouyang J, Xue Z, Wang M, He H, et al. Evaluating Large Language Models for Preoperative Patient Education in Superior Capsular Reconstruction: Comparative Study of Claude, GPT, and Gemini. JMIR Perioperative Medicine. 2025;8:e70047-e70047. doi:10.2196/70047

BibTeX
@article{Liu2025,
  author = {Liu, Yukang and Li, Hua and Ouyang, Jianfeng and Xue, Zhaowen and Wang, Min and He, Hebei and Song, Bin and Zheng, Xiaofei and Gan, Wenyi},
  title = {Evaluating Large Language Models for Preoperative Patient Education in Superior Capsular Reconstruction: Comparative Study of Claude, GPT, and Gemini},
  journal = {JMIR Perioperative Medicine},
  year = {2025},
  volume = {8},
  pages = {e70047--e70047},
  doi = {10.2196/70047}
}
Kostenlos starten →