Assessment of Quality and Readability of Information Provided by ChatGPT in Relation to Anterior Cruciate Ligament Injury
Journal of Personalized Medicine · 18.01.2024 · 14 (1) · 104
DOI: 10.3390/jpm14010104 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Personalized Medicine, online erschienen am 18.01.2024. Verfasst von Stephen Fahy und 4 weiteren Autor:innen. Beteiligte Einrichtung: Centrum für Muskuloskeletale Chirurgie, Charité Universitätsmedizin, 10117 Berlin, Germany. Die Arbeit wurde bisher 63-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The aim of our study was to evaluate the potential role of Artificial Intelligence tools like ChatGPT in patient education. To do this, we assessed both the quality and readability of information provided by ChatGPT 3.5 and 4 in relation to Anterior Cruciate Ligament (ACL) injury and treatment. ChatGPT 3.5 and 4 were used to answer common patient queries relating to ACL injuries and treatment. The quality of the information was assessed using the DISCERN criteria. Readability was assessed with the use of seven readability formulae: the Flesch–Kincaid Reading Grade Level, the Flesch Reading Ease Score, the Raygor Estimate, the SMOG, the Fry, the FORCAST, and the Gunning Fog. The mean reading grade level (RGL) was compared with the recommended 8th-grade reading level, the mean RGL among adults in America. The perceived quality and mean RGL of answers given by both ChatGPT 3.5 and 4 was also compared. Both ChatGPT 3.5 and 4 yielded DISCERN scores suggesting “good” quality of information, with ChatGPT 4 slightly outperforming 3.5. However, readability levels for both versions significantly exceeded the average 8th-grade reading level for American patients. ChatGPT 3.5 had a mean RGL of 18.08, while the mean RGL of ChatGPT 4 was 17.9, exceeding the average American reading grade level by 10.08 grade levels and 9.09 grade levels, respectively. While ChatGPT can provide both reliable and good quality information on ACL injuries and treatment options, the readability of the content may limit its utility. Additionally, the consistent lack of source citation represents a significant area of concern for patients and clinicians alike. If AI is to play a role in patient education, it must reliably produce information which is accurate, easily comprehensible, and clearly sourced.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Centrum für Muskuloskeletale Chirurgie, Charité Universitätsmedizin, 10117 Berlin, Germany.
ORCID-Profile: Stephen Fahy, Stephan Oehme
Zitieren
APA
Fahy, S., Oehme, S., Milinkovic, D., Jung, T., & Bartek, B. (2024). Assessment of Quality and Readability of Information Provided by ChatGPT in Relation to Anterior Cruciate Ligament Injury. Journal of Personalized Medicine, 14(1), 104. https://doi.org/10.3390/jpm14010104
Vancouver
Fahy S, Oehme S, Milinkovic D, Jung T, Bartek B. Assessment of Quality and Readability of Information Provided by ChatGPT in Relation to Anterior Cruciate Ligament Injury. Journal of Personalized Medicine. 2024;14(1):104. doi:10.3390/jpm14010104
BibTeX
@article{Fahy2024,
author = {Fahy, Stephen and Oehme, Stephan and Milinkovic, Danko and Jung, Tobias and Bartek, Benjamin},
title = {Assessment of Quality and Readability of Information Provided by ChatGPT in Relation to Anterior Cruciate Ligament Injury},
journal = {Journal of Personalized Medicine},
year = {2024},
volume = {14},
number = {1},
pages = {104},
doi = {10.3390/jpm14010104}
}
EUVIMED