A comparative analysis of readability, quality, and reliability in large language model outputs pertaining to knee osteoarthritis queries
Vergleich von Lesbarkeit, Qualität und Zuverlässigkeit von Large-Language-Model-Antworten zu Fragen zur Kniearthrose
PLOS One · 29.07.2026 · 21 (7) · e0353355
DOI: 10.1371/journal.pone.0353355 ↗Auf einen Blick
Antworten von Perplexity, ChatGPT-5 und Gemini zu Kniearthrose waren alle schwer verständlich. Perplexity erreichte die besten Werte für Qualität und Zuverlässigkeit.
Fragestellung
Wie unterscheiden sich Lesbarkeit, Genauigkeit, Qualität und Zuverlässigkeit von Chatbot-Antworten zu Kniearthrose?
Methode
Vergleichende Analyse von Antworten dreier KI-Chatbots auf 8 aus Google-Trends-Keywords abgeleitete Fragen; Bewertung mit CLI, ARI, LW, JAMA, mDISCERN, GQS und EQIP.
Ergebnisse
Alle drei Systeme lagen über dem Grade-6-Niveau, der Unterschied war signifikant (p < 0,05). ChatGPT-5 war am lesbarsten (FRES:45; GFOG:11,9; FKGL:9,24; CLI:14,03; SMOG:8,37; ARI:11,37; LW:7,2). Perplexity hatte höhere Qualitäts- und Zuverlässigkeitswerte (DISCERN: 4; JAMA: 2; GQS: 4; EQIP: 92,8).
Schlussfolgerung der Autor:innen
Die Autor:innen sehen bei populären KI-Chatbots für Patient:inneninformationen zu Kniearthrose erhebliche Probleme bei Verständlichkeit, Zuverlässigkeit und Inhaltsqualität.
Kernaussagen
- Alle drei Chatbots überschritten das empfohlene Verständlichkeitsniveau.
- ChatGPT-5 war am lesbarsten.
- Perplexity erzielte die besten Qualitäts- und Zuverlässigkeitswerte.
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Vergleichsstudie in der Zeitschrift PLOS One, online erschienen am 29.07.2026. Verfasst von Erdem Maraşlı und 2 weiteren Autor:innen. Beteiligte Einrichtung: Aydın State Hospital, Physical Medicine and Rehabilitation, Pain Medicine, Aydin, Turkey. Hauptthemen laut MEDLINE: Osteoarthritis, Knee, Comprehension. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This study aims to comparatively examine the readability, accuracy, and quality of responses provided by artificial intelligence (AI)-based chatbots such as Perplexity, ChatGPT-5, and Gemini to questions about knee osteoarthritis (KOA), which accounts for approximately four-fifths of the global osteoarthritis (OA) burden. In this study, 8 keywords were determined by excluding repetitive, irrelevant or synonymous ones from the 25 most frequently used English keywords associated with KOA based on Google Trends data, and these terms were asked as questions to three different artificial intelligence-based chatbots. The study measured readability using formulas like Coleman-Liau Index (CLI), Automated Readability Index (ARI), and Linsear Write (LW). Reliability of the information was assessed using the Journal of the American Medical Association (JAMA) benchmarks along with the modified DISCERN instrument. To determine overall content quality, the Global Quality Score (GQS) and the Ensuring Quality Information for Patients (EQIP) scale were applied. Together, these tools provided a comprehensive assessment of how understandable, reliable, and high-quality each chatbot’s responses were. The most frequently searched keywords related to OA were “osteoarthritis of knee,” “knee pain,” and “osteoarthritis knee pain.” A readability analysis of responses from three different AI-based chat systems revealed that all platforms had text levels above the Grade 6 threshold, and this difference was statistically significant (p < 0.05). Comparisons demonstrated that ChatGPT-5 produced the most readable content (FRES:45, GFOG:11.9, FKGL:9.24, CLI:14.03, SMOG:8.37, ARI:11.37, LW:7.2). However, Perplexity achieved significantly higher scores than ChatGPT-5 across all quality and reliability assessments, yielding superior median scores (DISCERN: 4, JAMA: 2, GQS: 4, EQIP: 92.8). Perplexity also outperformed Gemini in the mDISCERN reliability assessment (p = 0.001), while no significant difference in quality or reliability was found between Gemini and ChatGPT-5. No statistically significant difference was found between Gemini and ChatGPT in reliability and quality surveys. This analysis of KOA highlights significant challenges regarding the potential of popular AI chatbots for patient information. When examining readability levels, responses from these tools consistently exceed the recommended comprehensibility threshold, making it difficult for patients to absorb critical information. Furthermore, the relatively low scores recorded in reliability and content quality assessments raise significant concerns about the scientific validity and integrity of the medical information presented. Given these findings, the sufficient quality, robustness, and appropriate levels of understandability of future AI-based tools can only be ensured by the establishment and operation of an effective oversight mechanism.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Aydın State Hospital, Physical Medicine and Rehabilitation, Pain Medicine, Aydin, Turkey.
- Sivas Numune Hospital, Physical Medicine and Rehabilitation, Pain Medicine, Sivas, Turkey.
- Dokuz Eylul University, Anesthesiology and Reanimation, Critical Care Medicine, Izmir, Turkey.
ORCID-Profile: Erkan Ozduran
Zitieren
APA
Maraşlı, E., Ozduran, E., & Hancı, V. (2026). A comparative analysis of readability, quality, and reliability in large language model outputs pertaining to knee osteoarthritis queries. PLOS One, 21(7), e0353355. https://doi.org/10.1371/journal.pone.0353355
Vancouver
Maraşlı E, Ozduran E, Hancı V. A comparative analysis of readability, quality, and reliability in large language model outputs pertaining to knee osteoarthritis queries. PLOS One. 2026;21(7):e0353355. doi:10.1371/journal.pone.0353355
BibTeX
@article{Maral2026,
author = {Maraşlı, Erdem and Ozduran, Erkan and Hancı, Volkan},
title = {A comparative analysis of readability, quality, and reliability in large language model outputs pertaining to knee osteoarthritis queries},
journal = {PLOS One},
year = {2026},
volume = {21},
number = {7},
pages = {e0353355},
doi = {10.1371/journal.pone.0353355}
}
EUVIMED