EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Evaluating Artificial Intelligence in Orthopaedics: A Pilot Study on Accuracy and Reliability in Medical Student Competency Tests

Domy Pradana Putra, Farhan Ardiyanta Setyagisna, Heksa Trisnawati

eJournal Kedokteran Indonesia · 28.06.2026 · 14 (1) · 1

DOI: 10.23886/ejki.14.1229.1 ↗

Kurz gefasst

Fachartikel in der Zeitschrift eJournal Kedokteran Indonesia, online erschienen am 28.06.2026. Verfasst von Domy Pradana Putra und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Artificial intelligence (AI) is increasingly recognized as a valuable tool in medical education, yet its effectiveness across platforms remains underexplored. This study evaluated the performance of nine AI models-ChatGPT-4o, ChatGPT Mini, Gemini, Gemini Advanced, Perplexity, Perplexity Pro, Ortho Research Pro, Ortho AI, and Claude- in answering 30 expert-validated multiple-choice questions (MCQs) from the orthopaedics section of the UKMPPD. All models were evaluated concurrently between August and September 2024 using their official web interfaces. Each model was tested five times to assess accuracy and consistency. Statistical analysis was conducted using SPSS version 30.0. Normality and homogeneity were assessed using the Shapiro-Wilk and Levene’s tests. Accuracy differences were analyzed using oneway ANOVA followed by Tukey’s HSD post hoc test (p < 0.05). Reliability was evaluated using the intraclass correlation coefficient (ICC). Gemini demonstrated the highest mean accuracy (89 ± 2.79%), while ChatGPT Mini had the lowest (66 ± 3.33%). Significant differences in accuracy were observed (p < 0.05), with Gemini differing only from ChatGPT Mini and Perplexity. Most models demonstrated excellent reliability (ICC > 0.90), with Ortho Res. Pro among the best (ICC 0.956; 95% CI 0.925–0.977), while the remaining models showed good reliability. Overall, AI models showed strong potential for supporting medical exam preparation, although performance varied across platforms.

Zitieren

APA

Putra, D. P., Setyagisna, F. A., & Trisnawati, H. (2026). Evaluating Artificial Intelligence in Orthopaedics: A Pilot Study on Accuracy and Reliability in Medical Student Competency Tests. eJournal Kedokteran Indonesia, 14(1), 1. https://doi.org/10.23886/ejki.14.1229.1

Vancouver

Putra DP, Setyagisna FA, Trisnawati H. Evaluating Artificial Intelligence in Orthopaedics: A Pilot Study on Accuracy and Reliability in Medical Student Competency Tests. eJournal Kedokteran Indonesia. 2026;14(1):1. doi:10.23886/ejki.14.1229.1

BibTeX
@article{Putra2026,
  author = {Putra, Domy Pradana and Setyagisna, Farhan Ardiyanta and Trisnawati, Heksa},
  title = {Evaluating Artificial Intelligence in Orthopaedics: A Pilot Study on Accuracy and Reliability in Medical Student Competency Tests},
  journal = {eJournal Kedokteran Indonesia},
  year = {2026},
  volume = {14},
  number = {1},
  pages = {1},
  doi = {10.23886/ejki.14.1229.1}
}
Kostenlos starten →