EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Small Clinical Change, Big AI Decision? Metamorphic Testing of Large Language Models in Dental Clinical Decision-Making

Neşet Akay, Hüseyin Yanık, Fethi Atıl

Applied Sciences · 20.11.2026 · 16 (22) · 11505

DOI: 10.3390/app162211505 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 20.11.2026. Verfasst von Neşet Akay und 2 weiteren Autor:innen. Die Arbeit nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language model (LLM) chatbots are increasingly consulted for dental decisions, but case-by-case accuracy does not show whether their advice changes when, and only when, a clinically decisive fact changes. Metamorphic testing checks whether answers to two nearly identical cases relate as clinical rules require, without defining a single correct answer for each case. We applied it to oral and maxillofacial surgery and emergency dental decision-making. Thirty guideline-anchored relations were locked before querying, and 60 synthetic case pairs differing in one variable were submitted with a structured prompt to the ChatGPT (GPT-6 Astra) and Gemini (Gemini 3.8) web interfaces. Two blinded specialists scored each pair, with blinded adjudication of disagreements. In the main analysis (58 pairs; two defective vignettes withdrawn), relations were violated in 0 ChatGPT and 8 Gemini pairs (13.8%; p = 0.016, accounting for clustering within relations); with all 60 pairs, 1 vs. 8 (p = 0.063). Gemini violations clustered at numerical thresholds, escalation red flags and a sex-label invariance relation, and 13 of 116 Gemini responses, but no ChatGPT response, contained a safety violation. Chatbots that appear competent case by case can fail at clinically decisive thresholds; metamorphic testing offers a reproducible audit before clinical use.

Zitieren

APA

Akay, N., Yanık, H., & Atıl, F. (2026). Small Clinical Change, Big AI Decision? Metamorphic Testing of Large Language Models in Dental Clinical Decision-Making. Applied Sciences, 16(22), 11505. https://doi.org/10.3390/app162211505

Vancouver

Akay N, Yanık H, Atıl F. Small Clinical Change, Big AI Decision? Metamorphic Testing of Large Language Models in Dental Clinical Decision-Making. Applied Sciences. 2026;16(22):11505. doi:10.3390/app162211505

BibTeX
@article{Akay2026,
  author = {Akay, Neşet and Yanık, Hüseyin and Atıl, Fethi},
  title = {Small Clinical Change, Big AI Decision? Metamorphic Testing of Large Language Models in Dental Clinical Decision-Making},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {22},
  pages = {11505},
  doi = {10.3390/app162211505}
}
Kostenlos starten →