EUVIMEDOpdag EUVIMED
Crossref

Evaluating ChatGPT's Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis

Fan Ye, He Zhang, Xin Luo, Tong Wu, Qintai Yang, Zhaohui Shi

Otolaryngology–Head and Neck Surgery · 2024-08 · 171 (2) · 571-577

DOI: 10.1002/ohn.832 ↗

At a glance

Journal article in Otolaryngology–Head and Neck Surgery, published online on 2024-05-26. Written by Fan Ye and 5 co-authors. Institution: Department of Otolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China. Major MEDLINE topics: Sinusitis, Rhinitis, Allergic. Cited 20 times so far (Crossref); 17 references.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Objective

This study aims to evaluate the accuracy of ChatGPT in answering allergic rhinitis (AR) and chronic rhinosinusitis (CRS) related questions. Study Design This is a cross‐sectional study.

Setting

Each question was inputted as a separate, independent prompt.

Methods

Responses to AR (n = 189) and CRS (n = 242) related questions, generated by GPT‐3.5 and GPT‐4, were independently graded for accuracy by 2 senior rhinology professors, with disagreements adjudicated by a third reviewer.

Results

Overall, ChatGPT demonstrated a satisfactory performance, accurately answering over 80% of questions across all categories. Specifically, GPT‐4.0's accuracy in responding to AR‐related questions significantly exceeded that of GPT‐3.5, but distinction not evident in CRS‐related questions. Patient‐originated questions had a significantly higher accuracy compared to doctor‐originated questions when utilizing GPT‐4.0 to respond to AR‐related questions. This discrepancy was not observed with GPT‐3.5 or in the context of CRS‐related questions. Across different types of content, ChatGPT excelled in covering basic knowledge, prevention, and emotion for AR and CRS. However, it experienced challenges when addressing questions about recent advancements, a trend consistent across both GPT‐3.5 and GPT‐4.0 iterations. Importantly, the accuracy of responses remained unaffected when questions were posed in Chinese.

Conclusion

Our findings suggest ChatGPT's capability to convey accurate information for AR and CRS patients, and offer insights into its performance across various domains, guiding its utilization and improvement.

Subjects

MeSH terms (MEDLINE)

SinusitisRhinitis, AllergicHumansCross-Sectional StudiesChronic DiseaseSurveys and QuestionnairesMaleRhinitisFemalePatient Education as TopicRhinosinusitis

Author keywords

ChatGPTallergic rhinitisartificial intelligencechronic rhinosinusitis

Authors and institutions

  • Department of Otolaryngology-Head and Neck Surgery, The Third Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China.
  • Department of Allergy, The Third Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China.
  • Naso-Orbital-Maxilla and Skull Base Center, The Third Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China.
  • Key Laboratory of Airway Inflammatory Disease Research and Innovative Technology Translation, Guangzhou, China.

Cite

APA

Ye, F., Zhang, H., Luo, X., Wu, T., Yang, Q., & Shi, Z. (2024). Evaluating ChatGPT's Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis. Otolaryngology–Head and Neck Surgery, 171(2), 571-577. https://doi.org/10.1002/ohn.832

Vancouver

Ye F, Zhang H, Luo X, Wu T, Yang Q, Shi Z. Evaluating ChatGPT's Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis. Otolaryngology–Head and Neck Surgery. 2024;171(2):571-577. doi:10.1002/ohn.832

BibTeX
@article{Ye2024,
  author = {Ye, Fan and Zhang, He and Luo, Xin and Wu, Tong and Yang, Qintai and Shi, Zhaohui},
  title = {Evaluating ChatGPT's Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis},
  journal = {Otolaryngology–Head and Neck Surgery},
  year = {2024},
  volume = {171},
  number = {2},
  pages = {571--577},
  doi = {10.1002/ohn.832}
}
Start for free →