Evaluating ChatGPT's Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis
Otolaryngology–Head and Neck Surgery · 2024-08 · 171 (2) · 571-577
DOI: 10.1002/ohn.832 ↗Kurz gefasst
Fachartikel in der Zeitschrift Otolaryngology–Head and Neck Surgery, online erschienen am 26.05.2024. Verfasst von Fan Ye und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Otolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China. Hauptthemen laut MEDLINE: Sinusitis, Rhinitis, Allergic. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 17 Literaturangaben.
Abstract (Originalsprache)
Objective
This study aims to evaluate the accuracy of ChatGPT in answering allergic rhinitis (AR) and chronic rhinosinusitis (CRS) related questions. Study Design This is a cross‐sectional study.
Setting
Each question was inputted as a separate, independent prompt.
Methods
Responses to AR (n = 189) and CRS (n = 242) related questions, generated by GPT‐3.5 and GPT‐4, were independently graded for accuracy by 2 senior rhinology professors, with disagreements adjudicated by a third reviewer.
Results
Overall, ChatGPT demonstrated a satisfactory performance, accurately answering over 80% of questions across all categories. Specifically, GPT‐4.0's accuracy in responding to AR‐related questions significantly exceeded that of GPT‐3.5, but distinction not evident in CRS‐related questions. Patient‐originated questions had a significantly higher accuracy compared to doctor‐originated questions when utilizing GPT‐4.0 to respond to AR‐related questions. This discrepancy was not observed with GPT‐3.5 or in the context of CRS‐related questions. Across different types of content, ChatGPT excelled in covering basic knowledge, prevention, and emotion for AR and CRS. However, it experienced challenges when addressing questions about recent advancements, a trend consistent across both GPT‐3.5 and GPT‐4.0 iterations. Importantly, the accuracy of responses remained unaffected when questions were posed in Chinese.
Conclusion
Our findings suggest ChatGPT's capability to convey accurate information for AR and CRS patients, and offer insights into its performance across various domains, guiding its utilization and improvement.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Otolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China.
- Department of Allergy, The Third Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China.
- Naso-Orbital-Maxilla and Skull Base Center, The Third Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China.
- Key Laboratory of Airway Inflammatory Disease Research and Innovative Technology Translation, Guangzhou, China.
Zitieren
APA
Ye, F., Zhang, H., Luo, X., Wu, T., Yang, Q., & Shi, Z. (2024). Evaluating ChatGPT's Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis. Otolaryngology–Head and Neck Surgery, 171(2), 571-577. https://doi.org/10.1002/ohn.832
Vancouver
Ye F, Zhang H, Luo X, Wu T, Yang Q, Shi Z. Evaluating ChatGPT's Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis. Otolaryngology–Head and Neck Surgery. 2024;171(2):571-577. doi:10.1002/ohn.832
BibTeX
@article{Ye2024,
author = {Ye, Fan and Zhang, He and Luo, Xin and Wu, Tong and Yang, Qintai and Shi, Zhaohui},
title = {Evaluating ChatGPT's Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis},
journal = {Otolaryngology–Head and Neck Surgery},
year = {2024},
volume = {171},
number = {2},
pages = {571--577},
doi = {10.1002/ohn.832}
}
EUVIMED