EUVIMEDEUVIMED entdecken
Crossref

Assessing the accuracy and reproducibility of ChatGPT for responding to patient inquiries about otosclerosis

Utku Mete, Ömer Afşın Özmen

European Archives of Oto-Rhino-Laryngology · 2025-03 · 282 (3) · 1567-1575

DOI: 10.1007/s00405-024-09039-4 ↗

Kurz gefasst

Fachartikel in der Zeitschrift European Archives of Oto-Rhino-Laryngology, online erschienen am 26.10.2024. Verfasst von Utku Mete und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Otolaryngology-Head and Neck Surgery, Bursa Uludag University, Faculty of Medicine, Görükle Center Campus, Nilüfer, Bursa, 16059, Turke…. Hauptthemen laut MEDLINE: Otosclerosis, Internet. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 27 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: Patients increasingly use chatbots powered by artificial intelligence to seek information. However, there is a lack of reliable studies on the accuracy and reproducibility of the information provided by these models. Therefore, we conducted a study investigating the ChatGPT's responses to questions about otosclerosis. METHODS: 96 otosclerosis-related questions were collected from internet searches and websites of professional institutions and societies. Questions are divided into four sub-categories. These questions were directed at the latest version of ChatGPT Plus, and these responses were assessed by two otorhinolaryngology surgeons. Accuracy was graded as correct, incomplete, mixed, and irrelevant. Reproducibility was evaluated by comparing the consistency of the two answers to each specific question. RESULTS: The overall accuracy and reproducibility rates of GPT-4o for correct answers were found to be 64.60% and 89.60%, respectively. The findings showed correct answers for accuracy and reproducibility for basic knowledge were 64.70% and 91.20%; for diagnosis & management, 64.0% and 92.0%; for medical & surgical treatment, 52.95% and 82.35%; and for operative risks & postoperative period, 75.0% and 90.0%, respectively. There were no significant differences found between the answers and groups in terms of accuracy and reproducibility (p = 0.073 and p = 0.752, respectively). CONCLUSION: GPT-4o achieved satisfactory accuracy results, except in the diagnosis & management and medical & surgical treatment categories. Reproducibility was generally high across all categories. With the audio and visual communication capabilities of GPT-4o, under the supervision of a medical professional, this model can be utilized to provide medical information and support for otosclerosis patients.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

OtosclerosisInternetHumansReproducibility of ResultsSurveys and QuestionnairesGenerative Artificial Intelligence

Stichwörter der Autor:innen

Artificial intelligenceChatGPTHearing lossLarge language modelsOtosclerosisPatient information

Autor:innen und Institutionen

  • Department of Otolaryngology-Head and Neck Surgery, Bursa Uludag University, Faculty of Medicine, Görükle Center Campus, Nilüfer, Bursa, 16059, Turkey. drutkumete@gmail.com.
  • Department of Otolaryngology-Head and Neck Surgery, Bursa Uludag University, Faculty of Medicine, Görükle Center Campus, Nilüfer, Bursa, 16059, Turkey.

Zitieren

APA

Mete, U., & Özmen, Ö. A. (2025). Assessing the accuracy and reproducibility of ChatGPT for responding to patient inquiries about otosclerosis. European Archives of Oto-Rhino-Laryngology, 282(3), 1567-1575. https://doi.org/10.1007/s00405-024-09039-4

Vancouver

Mete U, Özmen ÖA. Assessing the accuracy and reproducibility of ChatGPT for responding to patient inquiries about otosclerosis. European Archives of Oto-Rhino-Laryngology. 2025;282(3):1567-1575. doi:10.1007/s00405-024-09039-4

BibTeX
@article{Mete2025,
  author = {Mete, Utku and Özmen, Ömer Afşın},
  title = {Assessing the accuracy and reproducibility of ChatGPT for responding to patient inquiries about otosclerosis},
  journal = {European Archives of Oto-Rhino-Laryngology},
  year = {2025},
  volume = {282},
  number = {3},
  pages = {1567--1575},
  doi = {10.1007/s00405-024-09039-4}
}
Kostenlos starten →