EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Assessment of Quality and Readability of Information Provided by ChatGPT in Relation to Anterior Cruciate Ligament Injury

Stephen Fahy iD, Stephan Oehme iD, Danko Milinkovic, Tobias Jung, Benjamin Bartek

Journal of Personalized Medicine · 18.01.2024 · 14 (1) · 104

DOI: 10.3390/jpm14010104 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Personalized Medicine, online erschienen am 18.01.2024. Verfasst von Stephen Fahy und 4 weiteren Autor:innen. Beteiligte Einrichtung: Centrum für Muskuloskeletale Chirurgie, Charité Universitätsmedizin, 10117 Berlin, Germany. Die Arbeit wurde bisher 63-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The aim of our study was to evaluate the potential role of Artificial Intelligence tools like ChatGPT in patient education. To do this, we assessed both the quality and readability of information provided by ChatGPT 3.5 and 4 in relation to Anterior Cruciate Ligament (ACL) injury and treatment. ChatGPT 3.5 and 4 were used to answer common patient queries relating to ACL injuries and treatment. The quality of the information was assessed using the DISCERN criteria. Readability was assessed with the use of seven readability formulae: the Flesch–Kincaid Reading Grade Level, the Flesch Reading Ease Score, the Raygor Estimate, the SMOG, the Fry, the FORCAST, and the Gunning Fog. The mean reading grade level (RGL) was compared with the recommended 8th-grade reading level, the mean RGL among adults in America. The perceived quality and mean RGL of answers given by both ChatGPT 3.5 and 4 was also compared. Both ChatGPT 3.5 and 4 yielded DISCERN scores suggesting “good” quality of information, with ChatGPT 4 slightly outperforming 3.5. However, readability levels for both versions significantly exceeded the average 8th-grade reading level for American patients. ChatGPT 3.5 had a mean RGL of 18.08, while the mean RGL of ChatGPT 4 was 17.9, exceeding the average American reading grade level by 10.08 grade levels and 9.09 grade levels, respectively. While ChatGPT can provide both reliable and good quality information on ACL injuries and treatment options, the readability of the content may limit its utility. Additionally, the consistent lack of source citation represents a significant area of concern for patients and clinicians alike. If AI is to play a role in patient education, it must reliably produce information which is accurate, easily comprehensible, and clearly sourced.

Schlagwörter

Stichwörter der Autor:innen

ACL reconstruction surgery (ACL-R)ChatGPTDISCERN criteriaanterior cruciate ligament (ACL)artificial intelligence (AI)health literacynatural language processingorthopaedic injuriespatient education materials (PEMS)readability

Autor:innen und Institutionen

  • Centrum für Muskuloskeletale Chirurgie, Charité Universitätsmedizin, 10117 Berlin, Germany.

ORCID-Profile: Stephen Fahy, Stephan Oehme

Zitieren

APA

Fahy, S., Oehme, S., Milinkovic, D., Jung, T., & Bartek, B. (2024). Assessment of Quality and Readability of Information Provided by ChatGPT in Relation to Anterior Cruciate Ligament Injury. Journal of Personalized Medicine, 14(1), 104. https://doi.org/10.3390/jpm14010104

Vancouver

Fahy S, Oehme S, Milinkovic D, Jung T, Bartek B. Assessment of Quality and Readability of Information Provided by ChatGPT in Relation to Anterior Cruciate Ligament Injury. Journal of Personalized Medicine. 2024;14(1):104. doi:10.3390/jpm14010104

BibTeX
@article{Fahy2024,
  author = {Fahy, Stephen and Oehme, Stephan and Milinkovic, Danko and Jung, Tobias and Bartek, Benjamin},
  title = {Assessment of Quality and Readability of Information Provided by ChatGPT in Relation to Anterior Cruciate Ligament Injury},
  journal = {Journal of Personalized Medicine},
  year = {2024},
  volume = {14},
  number = {1},
  pages = {104},
  doi = {10.3390/jpm14010104}
}
Kostenlos starten →