EUVIMEDEUVIMED entdecken
Mixed-Methods-StudieOpen Access

Educational Utility of Clinical Vignettes Generated in Japanese by ChatGPT-4: Mixed Methods Study

Hiromizu Takahashi iD, Kiyoshi Shikino iD, Takeshi Kondo iD, Akira Komori iD, Yuji Yamada iD, Mizue Saita iD, Toshio Naito iD

JMIR Medical Education · 13.08.2024 · 10 · e59133

DOI: 10.2196/59133 ↗

Kurz gefasst

Mixed-Methods-Studie in der Zeitschrift JMIR Medical Education, online erschienen am 13.08.2024. Verfasst von Hiromizu Takahashi und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of General Medicine, Juntendo University Faculty of Medicine, Tokyo, Japan. Hauptthemen laut MEDLINE: Artificial Intelligence. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Evaluating the accuracy and educational utility of artificial intelligence–generated medical cases, especially those produced by large language models such as ChatGPT-4 (developed by OpenAI), is crucial yet underexplored.

Objective

This study aimed to assess the educational utility of ChatGPT-4–generated clinical vignettes and their applicability in educational settings.

Methods

Using a convergent mixed methods design, a web-based survey was conducted from January 8 to 28, 2024, to evaluate 18 medical cases generated by ChatGPT-4 in Japanese. In the survey, 6 main question items were used to evaluate the quality of the generated clinical vignettes and their educational utility, which are information quality, information accuracy, educational usefulness, clinical match, terminology accuracy (TA), and diagnosis difficulty. Feedback was solicited from physicians specializing in general internal medicine or general medicine and experienced in medical education. Chi-square and Mann-Whitney U tests were performed to identify differences among cases, and linear regression was used to examine trends associated with physicians’ experience. Thematic analysis of qualitative feedback was performed to identify areas for improvement and confirm the educational utility of the cases.

Results

Of the 73 invited participants, 71 (97%) responded. The respondents, primarily male (64/71, 90%), spanned a broad range of practice years (from 1976 to 2017) and represented diverse hospital sizes throughout Japan. The majority deemed the information quality (mean 0.77, 95% CI 0.75-0.79) and information accuracy (mean 0.68, 95% CI 0.65-0.71) to be satisfactory, with these responses being based on binary data. The average scores assigned were 3.55 (95% CI 3.49-3.60) for educational usefulness, 3.70 (95% CI 3.65-3.75) for clinical match, 3.49 (95% CI 3.44-3.55) for TA, and 2.34 (95% CI 2.28-2.40) for diagnosis difficulty, based on a 5-point Likert scale. Statistical analysis showed significant variability in content quality and relevance across the cases (P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceHumansJapanMaleFemaleSurveys and QuestionnairesAdultEducation, MedicalClinical CompetenceEast Asian People

Stichwörter der Autor:innen

AIChatGPT-4JapanJapaneseartificial intelligenceclinical vignettesgenerative AImedical case generationmedical education

Autor:innen und Institutionen

  • Department of General Medicine, Juntendo University Faculty of Medicine, Tokyo, Japan.
  • Department of Community-Oriented Medical Education, Chiba University Graduate School of Medicine, Chiba, Japan.
  • Center for Postgraduate Clinical Training and Career Development, Nagoya University Hospital, Aichi, Japan.
  • Department of Emergency and Critical Care Medicine, Tsukuba Memorial Hospital, Tsukuba, Japan.
  • Brookdale Department of Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai, NY, NY, United States.

ORCID-Profile: Hiromizu Takahashi, Kiyoshi Shikino, Takeshi Kondo, Akira Komori, Yuji Yamada, Mizue Saita, Toshio Naito

Zitieren

APA

Takahashi, H., Shikino, K., Kondo, T., Komori, A., Yamada, Y., Saita, M., & Naito, T. (2024). Educational Utility of Clinical Vignettes Generated in Japanese by ChatGPT-4: Mixed Methods Study. JMIR Medical Education, 10, e59133. https://doi.org/10.2196/59133

Vancouver

Takahashi H, Shikino K, Kondo T, Komori A, Yamada Y, Saita M, et al. Educational Utility of Clinical Vignettes Generated in Japanese by ChatGPT-4: Mixed Methods Study. JMIR Medical Education. 2024;10:e59133. doi:10.2196/59133

BibTeX
@article{Takahashi2024,
  author = {Takahashi, Hiromizu and Shikino, Kiyoshi and Kondo, Takeshi and Komori, Akira and Yamada, Yuji and Saita, Mizue and Naito, Toshio},
  title = {Educational Utility of Clinical Vignettes Generated in Japanese by ChatGPT-4: Mixed Methods Study},
  journal = {JMIR Medical Education},
  year = {2024},
  volume = {10},
  pages = {e59133},
  doi = {10.2196/59133}
}
Kostenlos starten →