EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation

Nebiha Gozde Ispir, Gokalp Aslan iD, Gulsun Akay

Diagnostics · 21.10.2026 · 16 (20) · 3430

DOI: 10.3390/diagnostics16203430 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Diagnostics, online erschienen am 21.10.2026. Verfasst von Nebiha Gozde Ispir und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Oral and Dentomaxillofacial Radiology, Gazi University, Ankara 06490, Turkey. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background/Objectives: Artificial intelligence (AI) chatbots are increasingly used to obtain oral health information, yet prior dental studies have already evaluated overlapping domains such as accuracy, quality, readability, understandability, and actionability. This study applied and extended a multidimensional evaluation approach to compare AI-generated answers to dental patient questions, with emphasis on Turkish-language readability, standard PEMAT-P applicability, effect-size and uncertainty reporting, and inter-rater reliability.

Methods

Thirty-three AI-assisted, expert-curated questions were submitted to ChatGPT 5.2, Gemini, DeepSeek V3, and Grok 3, yielding 132 responses. Three dental evaluators independently assessed clinical accuracy, Global Quality Score (GQS), understandability (PEMAT-U), and actionability (PEMAT-A). PEMAT-P scores were calculated using the standard item structure and item-specific applicability rules, with items coded as not applicable (N/A) excluded from the denominator. Readability was calculated using a deterministic, rule-based implementation of the Ateşman formula. Model comparisons used the Friedman test, Bonferroni-adjusted Wilcoxon signed-rank tests, effect sizes, bootstrap confidence intervals, and two-way random-effects intraclass correlation coefficients. An exploratory Pareto analysis assessed multidimensional dominance without imposing arbitrary weights.

Results

Significant model differences were observed for all five dimensions (all p ≤ 0.002). Gemini had the highest mean GQS, clinical accuracy, PEMAT-U, and PEMAT-A scores, but its Ateşman readability score was significantly lower than those of ChatGPT, DeepSeek, and Grok. No significant readability differences were observed among the latter three models. Average-measure reliability was poor for GQS and clinical accuracy and moderate for PEMAT-U and PEMAT-A. No model dominated another across all five dimensions.

Conclusions

The chatbots demonstrated model-specific strengths rather than uniform superiority. Their performance should be interpreted multidimensionally, together with effect magnitude, uncertainty, and inter-rater reliability, rather than through a single overall ranking.

Schlagwörter

Stichwörter der Autor:innen

PEMATPareto analysisartificial intelligencedental patient educationeffect sizelarge language modelmultidimensional evaluationreadability

Autor:innen und Institutionen

  • Department of Oral and Dentomaxillofacial Radiology, Gazi University, Ankara 06490, Turkey.

ORCID-Profile: Gokalp Aslan

Zitieren

APA

Ispir, N. G., Aslan, G., & Akay, G. (2026). Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation. Diagnostics, 16(20), 3430. https://doi.org/10.3390/diagnostics16203430

Vancouver

Ispir NG, Aslan G, Akay G. Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation. Diagnostics. 2026;16(20):3430. doi:10.3390/diagnostics16203430

BibTeX
@article{Ispir2026,
  author = {Ispir, Nebiha Gozde and Aslan, Gokalp and Akay, Gulsun},
  title = {Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation},
  journal = {Diagnostics},
  year = {2026},
  volume = {16},
  number = {20},
  pages = {3430},
  doi = {10.3390/diagnostics16203430}
}
Kostenlos starten →