EUVIMEDObjevte EUVIMED
Crossref

Methodological Considerations in Evaluating Large Language Models for Anatomy Education

Ismail Sivri iD, Furkan Mehmet Ozden iD, Gamze Gul iD, Emre Kaygin iD, Tuncay Colak iD

Clinical Anatomy · 2026-06-27

DOI: 10.1002/ca.70170 ↗

At a glance

Journal article in Clinical Anatomy, published online on 2026-06-27. Written by Ismail Sivri and 4 co-authors. Institution: Department of Anatomy, Faculty of Medicine, Kocaeli University, İzmit, Kocaeli, Türkiye. 13 references.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Recent studies evaluating large language models (LLMs) in anatomy and medical education have provided valuable evidence regarding their potential role in examination‐based knowledge assessment, clinical education, and healthcare‐related decision support. Assessing large language models with non‐English anatomy questions is important, given the predominance of English‐language datasets in the current literature. However, comparative LLM studies also raise broader methodological concerns that may affect reproducibility, fairness, and interpretation. In this letter, we highlight three major categories of methodological factors to consider in future LLM evaluation studies: technical factors, session‐related factors, and experimental design factors. Technical factors include model version, interface type, access date, browser, device, operating system, internet connection, and timing method. Session‐related factors include memory, personalization, chat history, custom instructions, and the use of sequential versus independent testing sessions. Experimental design factors include dataset source, question selection, exact prompt wording, input language, response format, number of attempts, question order, scoring procedures, expert evaluation, and inter‐rater agreement. We suggest that future LLM comparison studies in anatomy, education, and clinical fields should provide detailed methodological reporting and align with emerging frameworks such as TRIPOD‐LLM, CONSORT‐AI, and MI‐CLEAR‐LLM. Improved transparency in these areas will strengthen reproducibility and comparability across studies.

Subjects

Author keywords

ChatGPTartificial intelligencedata leakagelarge language modelsmedical educationmemory biasreporting guidelinesreproducibility

Authors and institutions

  • Department of Anatomy, Faculty of Medicine, Kocaeli University, İzmit, Kocaeli, Türkiye.

ORCID profiles: Ismail Sivri, Furkan Mehmet Ozden, Gamze Gul, Emre Kaygin, Tuncay Colak

Cite

APA

Sivri, I., Ozden, F. M., Gul, G., Kaygin, E., & Colak, T. (2026). Methodological Considerations in Evaluating Large Language Models for Anatomy Education. Clinical Anatomy. https://doi.org/10.1002/ca.70170

Vancouver

Sivri I, Ozden FM, Gul G, Kaygin E, Colak T. Methodological Considerations in Evaluating Large Language Models for Anatomy Education. Clinical Anatomy. 2026. doi:10.1002/ca.70170

BibTeX
@article{Sivri2026,
  author = {Sivri, Ismail and Ozden, Furkan Mehmet and Gul, Gamze and Kaygin, Emre and Colak, Tuncay},
  title = {Methodological Considerations in Evaluating Large Language Models for Anatomy Education},
  journal = {Clinical Anatomy},
  year = {2026},
  doi = {10.1002/ca.70170}
}
Start for free →