EUVIMEDEUVIMED entdecken
Crossref

Comparative evaluation of large language models in treatment decision-making for skeletal class II malocclusion using clinical and cephalometric prompts

Beril Erdil, Samet Özden

Journal of Health Sciences and Medicine · 25.10.2026 · 9 (6) · 1726-1734

DOI: 10.32322/jhsm.1991896 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Health Sciences and Medicine, online erschienen am 25.10.2026. Verfasst von Beril Erdil und 1 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Gefördert von: This study received no financial support..

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aims

To evaluate the performance of ChatGPT, Gemini, and Grok in selecting among camouflage treatment, functional treatment, and orthognathic surgery for patients with skeletal class II malocclusion and to compare their classifications with expert orthodontist consensus.

Methods

This retrospective comparative decision-accuracy study included a balanced sample of 45 patients, comprising 15 cases in each treatment category based on the unanimous agreement of three specialist orthodontists. Standardized scenarios containing anonymized demographic, clinical, and cephalometric data were submitted independently to the three large language models. Overall accuracy, class-specific F1 scores, and Cohen’s kappa coefficients were calculated. Differences in accuracy were evaluated using Cochran’s Q and exact McNemar tests with Bonferroni correction. The association between age and prediction accuracy was assessed using the Mann-Whitney U test.

Results

Overall accuracy was 77.8% for Grok, 75.6% for ChatGPT, and 68.9% for Gemini. Grok and ChatGPT showed substantial agreement with expert consensus, with kappa values of 0.667 and 0.633, respectively, whereas Gemini showed moderate agreement (κ=0.533). All models achieved their highest F1 scores for functional treatment (0.929-1.000). Gemini frequently classified camouflage cases as requiring orthognathic surgery, whereas Grok tended to classify surgical cases as suitable for camouflage. Overall accuracy did not differ significantly among the models (Q=1.300; p=0.522). Incorrect ChatGPT predictions were associated with a higher age rank than correct predictions (p=0.006).

Conclusion

The models demonstrated moderate to substantial agreement with expert orthodontists and performed particularly well in functional treatment cases. However, errors in distinguishing camouflage treatment from orthognathic surgery indicate that LLM-based classification should be interpreted cautiously and should not replace comprehensive clinical assessment.

Förderung

  • This study received no financial support.

Zitieren

APA

Erdil, B., & Özden, S. (2026). Comparative evaluation of large language models in treatment decision-making for skeletal class II malocclusion using clinical and cephalometric prompts. Journal of Health Sciences and Medicine, 9(6), 1726-1734. https://doi.org/10.32322/jhsm.1991896

Vancouver

Erdil B, Özden S. Comparative evaluation of large language models in treatment decision-making for skeletal class II malocclusion using clinical and cephalometric prompts. Journal of Health Sciences and Medicine. 2026;9(6):1726-1734. doi:10.32322/jhsm.1991896

BibTeX
@article{Erdil2026,
  author = {Erdil, Beril and Özden, Samet},
  title = {Comparative evaluation of large language models in treatment decision-making for skeletal class II malocclusion using clinical and cephalometric prompts},
  journal = {Journal of Health Sciences and Medicine},
  year = {2026},
  volume = {9},
  number = {6},
  pages = {1726--1734},
  doi = {10.32322/jhsm.1991896}
}
Kostenlos starten →