EUVIMEDEUVIMED entdecken
Vergleichsstudie

Using large language models ( ChatGPT , Copilot, PaLM , Bard, and Gemini) in Gross Anatomy course: Comparative analysis

Volodymyr Mavrych, Paul Ganguly, Olena Bolgova iD

Clinical Anatomy · 2025-03 · 38 (2) · 200-210

DOI: 10.1002/ca.24244 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Clinical Anatomy, online erschienen am 21.11.2024. Verfasst von Volodymyr Mavrych und 2 weiteren Autor:innen. Beteiligte Einrichtung: College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia. Hauptthemen laut MEDLINE: Anatomy, Artificial Intelligence, Education, Medical, Undergraduate. Die Arbeit wurde bisher 62-mal zitiert (Crossref) und nennt 33 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The increasing application of generative artificial intelligence large language models (LLMs) in various fields, including medical education, raises questions about their accuracy. The primary aim of our study was to undertake a detailed comparative analysis of the proficiencies and accuracies of six different LLMs (ChatGPT‐4, ChatGPT‐3.5‐turbo, ChatGPT‐3.5, Copilot, PaLM, Bard, and Gemini) in responding to medical multiple‐choice questions (MCQs), and in generating clinical scenarios and MCQs for upper limb topics in a Gross Anatomy course for medical students. Selected chatbots were tested, answering 50 USMLE‐style MCQs. The questions were randomly selected from the Gross Anatomy course exam database for medical students and reviewed by three independent experts. The results of five successive attempts to answer each set of questions by the chatbots were evaluated in terms of accuracy, relevance, and comprehensiveness. The best result was provided by ChatGPT‐4, which answered 60.5% ± 1.9% of questions accurately, then Copilot (42.0% ± 0.0%) and ChatGPT‐3.5 (41.0% ± 5.3%), followed by ChatGPT‐3.5‐turbo (38.5% ± 5.7%). Google PaLM 2 (34.5% ± 4.4%) and Bard (33.5% ± 3.0%) gave the poorest results. The overall performance of GPT‐4 was statistically superior ( p < 0.05) to those of Copilot, GPT‐3.5, GPT‐Turbo, PaLM2, and Bard by 18.6%, 19.5%, 22%, 26%, and 27%, respectively. Each chatbot was then asked to generate a clinical scenario for each of the three randomly selected topics—anatomical snuffbox, supracondylar fracture of the humerus, and the cubital fossa—and three related anatomical MCQs with five options each, and to indicate the correct answers. Two independent experts analyzed and graded 216 records received (0–5 scale). The best results were recorded for ChatGPT‐4, then for Gemini, ChatGPT‐3.5, and ChatGPT‐3.5‐turbo, Copilot, followed by Google PaLM 2; Copilot had the lowest grade. Technological progress notwithstanding, LLMs have yet to mature sufficiently to take over the role of teacher or facilitator completely within a Gross Anatomy course; however, they can be valuable tools for medical educators.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AnatomyArtificial IntelligenceEducation, Medical, UndergraduateHumansEducational MeasurementGenerative Artificial IntelligenceLarge Language Models

Stichwörter der Autor:innen

ChatGPTGeminiPaLManatomyartificial intelligencebardcopilotextremitieslarge language models

Autor:innen und Institutionen

  • College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia.

ORCID-Profile: Olena Bolgova

Zitieren

APA

Mavrych, V., Ganguly, P., & Bolgova, O. (2025). Using large language models ( ChatGPT , Copilot, PaLM , Bard, and Gemini) in Gross Anatomy course: Comparative analysis. Clinical Anatomy, 38(2), 200-210. https://doi.org/10.1002/ca.24244

Vancouver

Mavrych V, Ganguly P, Bolgova O. Using large language models ( ChatGPT , Copilot, PaLM , Bard, and Gemini) in Gross Anatomy course: Comparative analysis. Clinical Anatomy. 2025;38(2):200-210. doi:10.1002/ca.24244

BibTeX
@article{Mavrych2025,
  author = {Mavrych, Volodymyr and Ganguly, Paul and Bolgova, Olena},
  title = {Using large language models ( ChatGPT , Copilot, PaLM , Bard, and Gemini) in Gross Anatomy course: Comparative analysis},
  journal = {Clinical Anatomy},
  year = {2025},
  volume = {38},
  number = {2},
  pages = {200--210},
  doi = {10.1002/ca.24244}
}
Kostenlos starten →