EUVIMEDEUVIMED entdecken
Open Access

Assessment of Large Language Model Performance on Medical School Essay-Style Concept Appraisal Questions: Exploratory Study

Seysha Mehta iD, Eliot N Haddad iD, Indira Bhavsar Burke iD, Alana K Majors iD, Rie Maeda iD, Sean M Burke iD, Abhishek Deshpande iD, Amy S Nowacki iD, Christina C Lindenmeyer iD, Neil Mehta iD

JMIR Medical Education · 16.06.2025 · 11 · e72034-e72034

DOI: 10.2196/72034 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Medical Education, online erschienen am 16.06.2025. Verfasst von Seysha Mehta und 9 weiteren Autor:innen. Beteiligte Einrichtung: Cleveland Clinic Lerner College of Medicine, School of Medicine, Case Western Reserve University, 9500 Euclid Ave, G10, Cleveland, OH, 44195, United …. Hauptthemen laut MEDLINE: Students, Medical, Language, Educational Measurement, Education, Medical, Undergraduate. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 10 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Bing Chat (subsequently renamed Microsoft Copilot)—a ChatGPT 4.0–based large language model—demonstrated comparable performance to medical students in answering essay-style concept appraisals, while assessors struggled to differentiate artificial intelligence (AI) responses from human responses. These results highlight the need to prepare students and educators for a future world of AI by fostering reflective learning practices and critical thinking.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Students, MedicalLanguageEducational MeasurementEducation, Medical, UndergraduateHumansArtificial IntelligenceSchools, MedicalLarge Language Models

Stichwörter der Autor:innen

Microsoft Copilotartificial intelligenceessay-type questionsgenerative AIlarge language models

Autor:innen und Institutionen

  • Cleveland Clinic Lerner College of Medicine, School of Medicine, Case Western Reserve University, 9500 Euclid Ave, G10, Cleveland, OH, 44195, United States, 1 2164456512, 1 2164451007.
  • Department of Internal Medicine, The University of Texas Southwestern Medical Center, Dallas, TX, United States.

ORCID-Profile: Seysha Mehta, Eliot N Haddad, Indira Bhavsar Burke, Alana K Majors, Rie Maeda, Sean M Burke, Abhishek Deshpande, Amy S Nowacki, Christina C Lindenmeyer, Neil Mehta

Zitieren

APA

Mehta, S., Haddad, E. N., Burke, I. B., Majors, A. K., Maeda, R., Burke, S. M., Deshpande, A., Nowacki, A. S., Lindenmeyer, C. C., & Mehta, N. (2025). Assessment of Large Language Model Performance on Medical School Essay-Style Concept Appraisal Questions: Exploratory Study. JMIR Medical Education, 11, e72034-e72034. https://doi.org/10.2196/72034

Vancouver

Mehta S, Haddad EN, Burke IB, Majors AK, Maeda R, Burke SM, et al. Assessment of Large Language Model Performance on Medical School Essay-Style Concept Appraisal Questions: Exploratory Study. JMIR Medical Education. 2025;11:e72034-e72034. doi:10.2196/72034

BibTeX
@article{Mehta2025,
  author = {Mehta, Seysha and Haddad, Eliot N and Burke, Indira Bhavsar and Majors, Alana K and Maeda, Rie and Burke, Sean M and Deshpande, Abhishek and Nowacki, Amy S and Lindenmeyer, Christina C and Mehta, Neil},
  title = {Assessment of Large Language Model Performance on Medical School Essay-Style Concept Appraisal Questions: Exploratory Study},
  journal = {JMIR Medical Education},
  year = {2025},
  volume = {11},
  pages = {e72034--e72034},
  doi = {10.2196/72034}
}
Kostenlos starten →