EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating the scientific reliability of ChatGPT as a source of information on asthma

Simon Høj, Simon Francis Thomsen, Charlotte Suppli Ulrik, Hanieh Meteran, Torben Sigsgaard, Howraman Meteran

Journal of Allergy and Clinical Immunology: Global · 2024-11 · 3 (4) · 100330

DOI: 10.1016/j.jacig.2024.100330 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Allergy and Clinical Immunology: Global, 2024-11. Verfasst von Simon Høj und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Dermatology, Venereology, and Wound Healing Centre, Copenhagen University Hospital-Bispebjerg, Bispebjerg, Denmark. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 9 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

This study assessed the reliability of ChatGPT as a source of information on asthma, given the increasing use of artificial intelligence-driven models for medical information. Prior concerns about misinformation on atopic diseases in various digital platforms underline the importance of this evaluation.

Objective

We aimed to evaluate the scientific reliability of ChatGPT as a source of information on asthma.

Methods

The study involved analyzing ChatGPT's responses to 26 asthma-related questions, each followed by a follow-up question. These encompassed definition/risk factors, diagnosis, treatment, lifestyle factors, and specific clinical inquiries. Medical professionals specialized in allergic and respiratory diseases independently assessed the responses using a 1-to-5 accuracy scale.

Results

Approximately 81% of the responses scored 4 or higher, suggesting a generally high accuracy level. However, 5 responses scored >3, indicating minor potentially harmful inaccuracies. The overall median score was 4. Fleiss multirater kappa value showed moderate agreement among raters.

Conclusion

ChatGPT generally provides reliable asthma-related information, but its limitations, such as lack of depth in certain responses and inability to cite sources or update in real time, were noted. It shows promise as an educational tool, but it should not be a substitute for professional medical advice. Future studies should explore its applicability for different user demographics and compare it with newer artificial intelligence models.

Schlagwörter

Stichwörter der Autor:innen

AIAsthmaChatGPTartificial intelligencepatient education

Autor:innen und Institutionen

  • Department of Dermatology, Venereology, and Wound Healing Centre, Copenhagen University Hospital-Bispebjerg, Bispebjerg, Denmark.
  • Department of Public Health, Environment, Occupation, and Health, Aarhus University, Aarhus, Denmark.
  • Department of Biomedical Sciences, University of Copenhagen, Copenhagen, Denmark.
  • Department of Respiratory Medicine, Copenhagen University Hospital-Hvidovre, Hvidovre, Denmark.
  • Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark.
  • Department of Internal Medicine, Section of Endocrinology, Copenhagen University Hospital-Hvidovre, Hvidovre, Denmark.

Zitieren

APA

Høj, S., Thomsen, S. F., Ulrik, C. S., Meteran, H., Sigsgaard, T., & Meteran, H. (2024). Evaluating the scientific reliability of ChatGPT as a source of information on asthma. Journal of Allergy and Clinical Immunology: Global, 3(4), 100330. https://doi.org/10.1016/j.jacig.2024.100330

Vancouver

Høj S, Thomsen SF, Ulrik CS, Meteran H, Sigsgaard T, Meteran H. Evaluating the scientific reliability of ChatGPT as a source of information on asthma. Journal of Allergy and Clinical Immunology: Global. 2024;3(4):100330. doi:10.1016/j.jacig.2024.100330

BibTeX
@article{Hj2024,
  author = {Høj, Simon and Thomsen, Simon Francis and Ulrik, Charlotte Suppli and Meteran, Hanieh and Sigsgaard, Torben and Meteran, Howraman},
  title = {Evaluating the scientific reliability of ChatGPT as a source of information on asthma},
  journal = {Journal of Allergy and Clinical Immunology: Global},
  year = {2024},
  volume = {3},
  number = {4},
  pages = {100330},
  doi = {10.1016/j.jacig.2024.100330}
}
Kostenlos starten →