EUVIMEDEUVIMED entdecken
Vergleichsstudie

ChatGPT versus a customized AI chatbot (Anatbuddy) for anatomy education: A comparative pilot study

Gautham Arun, Vivek Perumal, Francis Paul John Bato Urias, Yan En Ler, Bryan Wen Tao Tan, Ranganath Vallabhajosyula, Emmanuel Tan, Olivia Ng iD, Kian Bee Ng, Sreenivasulu Reddy Mogali iD

Anatomical Sciences Education · 2024-10 · 17 (7) · 1396-1405

DOI: 10.1002/ase.2502 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Anatomical Sciences Education, online erschienen am 21.08.2024. Verfasst von Gautham Arun und 9 weiteren Autor:innen. Beteiligte Einrichtung: Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore. Hauptthemen laut MEDLINE: Anatomy. Die Arbeit wurde bisher 64-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: Special Project Grant from the Medical Education, Research and Scholarship Unit, Lee Kong Chian School of Medicine.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) have the potential to improve education by personalizing learning. However, ChatGPT‐generated content has been criticized for sometimes producing false, biased, and/or hallucinatory information. To evaluate AI's ability to return clear and accurate anatomy information, this study generated a custom interactive and intelligent chatbot (Anatbuddy) through an Open AI Application Programming Interface (API) that enables seamless AI‐driven interactions within a secured cloud infrastructure. Anatbuddy was programmed through a Retrieval Augmented Generation (RAG) method to provide context‐aware responses to user queries based on a predetermined knowledge base. To compare their outputs, various queries (i.e., prompts) on thoracic anatomy ( n = 18) were fed into Anatbuddy and ChatGPT 3.5. A panel comprising three experienced anatomists evaluated both tools' responses for factual accuracy, relevance, completeness, coherence, and fluency on a 5‐point Likert scale. These ratings were reviewed by a third party blinded to the study, who revised and finalized scores as needed. Anatbuddy's factual accuracy (mean ± SD = 4.78/5.00 ± 0.43; median = 5.00) was rated significantly higher ( U = 84, p = 0.01) than ChatGPT's accuracy (4.11 ± 0.83; median = 4.00). No statistically significant differences were detected between the chatbots for the other variables. Given ChatGPT's current content knowledge limitations, we strongly recommend the anatomy profession develop a custom AI chatbot for anatomy education utilizing a carefully curated knowledge base to ensure accuracy. Further research is needed to determine students' acceptance of custom chatbots for anatomy education and their influence on learning experiences and outcomes.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AnatomyPilot ProjectsHumansArtificial IntelligenceComputer-Assisted InstructionEducational MeasurementLearningSoftware

Stichwörter der Autor:innen

ChatGPTanatomical educationartificial intelligencechatbotpersonalized learningretrieval augmented generation

Autor:innen und Institutionen

  • Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore.
  • Singapore Polytechnic, Singapore, Singapore.

ORCID-Profile: Olivia Ng, Sreenivasulu Reddy Mogali

Förderung

  • Special Project Grant from the Medical Education, Research and Scholarship Unit, Lee Kong Chian School of Medicine

Zitieren

APA

Arun, G., Perumal, V., Urias, F. P. J. B., Ler, Y. E., Tan, B. W. T., Vallabhajosyula, R., Tan, E., Ng, O., Ng, K. B., & Mogali, S. R. (2024). ChatGPT versus a customized AI chatbot (Anatbuddy) for anatomy education: A comparative pilot study. Anatomical Sciences Education, 17(7), 1396-1405. https://doi.org/10.1002/ase.2502

Vancouver

Arun G, Perumal V, Urias FPJB, Ler YE, Tan BWT, Vallabhajosyula R, et al. ChatGPT versus a customized AI chatbot (Anatbuddy) for anatomy education: A comparative pilot study. Anatomical Sciences Education. 2024;17(7):1396-1405. doi:10.1002/ase.2502

BibTeX
@article{Arun2024,
  author = {Arun, Gautham and Perumal, Vivek and Urias, Francis Paul John Bato and Ler, Yan En and Tan, Bryan Wen Tao and Vallabhajosyula, Ranganath and Tan, Emmanuel and Ng, Olivia and Ng, Kian Bee and Mogali, Sreenivasulu Reddy},
  title = {ChatGPT versus a customized AI chatbot (Anatbuddy) for anatomy education: A comparative pilot study},
  journal = {Anatomical Sciences Education},
  year = {2024},
  volume = {17},
  number = {7},
  pages = {1396--1405},
  doi = {10.1002/ase.2502}
}
Kostenlos starten →