Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation
Diagnostics · 21.10.2026 · 16 (20) · 3430
DOI: 10.3390/diagnostics16203430 ↗Kurz gefasst
Fachartikel in der Zeitschrift Diagnostics, online erschienen am 21.10.2026. Verfasst von Nebiha Gozde Ispir und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Oral and Dentomaxillofacial Radiology, Gazi University, Ankara 06490, Turkey. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background/Objectives: Artificial intelligence (AI) chatbots are increasingly used to obtain oral health information, yet prior dental studies have already evaluated overlapping domains such as accuracy, quality, readability, understandability, and actionability. This study applied and extended a multidimensional evaluation approach to compare AI-generated answers to dental patient questions, with emphasis on Turkish-language readability, standard PEMAT-P applicability, effect-size and uncertainty reporting, and inter-rater reliability.
Methods
Thirty-three AI-assisted, expert-curated questions were submitted to ChatGPT 5.2, Gemini, DeepSeek V3, and Grok 3, yielding 132 responses. Three dental evaluators independently assessed clinical accuracy, Global Quality Score (GQS), understandability (PEMAT-U), and actionability (PEMAT-A). PEMAT-P scores were calculated using the standard item structure and item-specific applicability rules, with items coded as not applicable (N/A) excluded from the denominator. Readability was calculated using a deterministic, rule-based implementation of the Ateşman formula. Model comparisons used the Friedman test, Bonferroni-adjusted Wilcoxon signed-rank tests, effect sizes, bootstrap confidence intervals, and two-way random-effects intraclass correlation coefficients. An exploratory Pareto analysis assessed multidimensional dominance without imposing arbitrary weights.
Results
Significant model differences were observed for all five dimensions (all p ≤ 0.002). Gemini had the highest mean GQS, clinical accuracy, PEMAT-U, and PEMAT-A scores, but its Ateşman readability score was significantly lower than those of ChatGPT, DeepSeek, and Grok. No significant readability differences were observed among the latter three models. Average-measure reliability was poor for GQS and clinical accuracy and moderate for PEMAT-U and PEMAT-A. No model dominated another across all five dimensions.
Conclusions
The chatbots demonstrated model-specific strengths rather than uniform superiority. Their performance should be interpreted multidimensionally, together with effect magnitude, uncertainty, and inter-rater reliability, rather than through a single overall ranking.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Oral and Dentomaxillofacial Radiology, Gazi University, Ankara 06490, Turkey.
ORCID-Profile: Gokalp Aslan
Zitieren
APA
Ispir, N. G., Aslan, G., & Akay, G. (2026). Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation. Diagnostics, 16(20), 3430. https://doi.org/10.3390/diagnostics16203430
Vancouver
Ispir NG, Aslan G, Akay G. Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation. Diagnostics. 2026;16(20):3430. doi:10.3390/diagnostics16203430
BibTeX
@article{Ispir2026,
author = {Ispir, Nebiha Gozde and Aslan, Gokalp and Akay, Gulsun},
title = {Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation},
journal = {Diagnostics},
year = {2026},
volume = {16},
number = {20},
pages = {3430},
doi = {10.3390/diagnostics16203430}
}
EUVIMED