Comparative evaluation of large language models in treatment decision-making for skeletal class II malocclusion using clinical and cephalometric prompts
Journal of Health Sciences and Medicine · 25.10.2026 · 9 (6) · 1726-1734
DOI: 10.32322/jhsm.1991896 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Health Sciences and Medicine, online erschienen am 25.10.2026. Verfasst von Beril Erdil und 1 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Gefördert von: This study received no financial support..
Abstract (Originalsprache)
Aims
To evaluate the performance of ChatGPT, Gemini, and Grok in selecting among camouflage treatment, functional treatment, and orthognathic surgery for patients with skeletal class II malocclusion and to compare their classifications with expert orthodontist consensus.
Methods
This retrospective comparative decision-accuracy study included a balanced sample of 45 patients, comprising 15 cases in each treatment category based on the unanimous agreement of three specialist orthodontists. Standardized scenarios containing anonymized demographic, clinical, and cephalometric data were submitted independently to the three large language models. Overall accuracy, class-specific F1 scores, and Cohen’s kappa coefficients were calculated. Differences in accuracy were evaluated using Cochran’s Q and exact McNemar tests with Bonferroni correction. The association between age and prediction accuracy was assessed using the Mann-Whitney U test.
Results
Overall accuracy was 77.8% for Grok, 75.6% for ChatGPT, and 68.9% for Gemini. Grok and ChatGPT showed substantial agreement with expert consensus, with kappa values of 0.667 and 0.633, respectively, whereas Gemini showed moderate agreement (κ=0.533). All models achieved their highest F1 scores for functional treatment (0.929-1.000). Gemini frequently classified camouflage cases as requiring orthognathic surgery, whereas Grok tended to classify surgical cases as suitable for camouflage. Overall accuracy did not differ significantly among the models (Q=1.300; p=0.522). Incorrect ChatGPT predictions were associated with a higher age rank than correct predictions (p=0.006).
Conclusion
The models demonstrated moderate to substantial agreement with expert orthodontists and performed particularly well in functional treatment cases. However, errors in distinguishing camouflage treatment from orthognathic surgery indicate that LLM-based classification should be interpreted cautiously and should not replace comprehensive clinical assessment.
Förderung
- This study received no financial support.
Zitieren
APA
Erdil, B., & Özden, S. (2026). Comparative evaluation of large language models in treatment decision-making for skeletal class II malocclusion using clinical and cephalometric prompts. Journal of Health Sciences and Medicine, 9(6), 1726-1734. https://doi.org/10.32322/jhsm.1991896
Vancouver
Erdil B, Özden S. Comparative evaluation of large language models in treatment decision-making for skeletal class II malocclusion using clinical and cephalometric prompts. Journal of Health Sciences and Medicine. 2026;9(6):1726-1734. doi:10.32322/jhsm.1991896
BibTeX
@article{Erdil2026,
author = {Erdil, Beril and Özden, Samet},
title = {Comparative evaluation of large language models in treatment decision-making for skeletal class II malocclusion using clinical and cephalometric prompts},
journal = {Journal of Health Sciences and Medicine},
year = {2026},
volume = {9},
number = {6},
pages = {1726--1734},
doi = {10.32322/jhsm.1991896}
}
EUVIMED