ChatENT: Augmented Large Language Model for Expert Knowledge Retrieval in Otolaryngology–Head and Neck Surgery
Otolaryngology–Head and Neck Surgery · 2024-10 · 171 (4) · 1042-1051
DOI: 10.1002/ohn.864 ↗At a glance
Journal article in Otolaryngology–Head and Neck Surgery, published online on 2024-06-19. Written by Cai Long and 10 co-authors. Institution: Division of Otolaryngology-Head and Neck Surgery, University of Alberta, Edmonton, Alberta, Canada. Major MEDLINE topics: Otolaryngology. Cited 35 times so far (Crossref); 35 references. The full text is freely available (open access, CC BY-NC 4.0).
Abstract (original language)
Objective
The recent surge in popularity of large language models (LLMs), such as ChatGPT, has showcased their proficiency in medical examinations and potential applications in health care. However, LLMs possess inherent limitations, including inconsistent accuracy, specific prompting requirements, and the risk of generating harmful hallucinations. A domain‐specific model might address these limitations effectively. Study Design Developmental design.
Setting
Virtual.
Methods
Otolaryngology–head and neck surgery (OHNS) relevant data were systematically gathered from open‐access Internet sources and indexed into a knowledge database. We leveraged Retrieval‐Augmented Language Modeling to recall this information and utilized it for pretraining, which was then integrated into ChatGPT4.0, creating an OHNS‐specific knowledge question & answer platform known as ChatENT. The model is further tested on different types of questions.
Results
ChatENT showed enhanced performance in the analysis and interpretation of OHNS information, outperforming ChatGPT4.0 in both the Canadian Royal College OHNS sample examination questions challenge and the US board practice questions challenge, with a 58.4% and 26.0% error reduction, respectively. ChatENT generated fewer hallucinations and demonstrated greater consistency.
Conclusion
To the best of our knowledge, ChatENT is the first specialty‐specific knowledge retrieval artificial intelligence in the medical field that utilizes the latest LLM. It appears to have considerable promise in areas such as medical education, patient education, and clinical decision support. The model has demonstrated the capacity to overcome the limitations of existing LLMs, thereby signaling a future of more precise, safe, and user‐friendly applications in the realm of OHNS and other medical fields.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Division of Otolaryngology-Head and Neck Surgery, University of Alberta, Edmonton, Alberta, Canada.
- Copula AI, New York, New York, USA.
- Faculty of Medicine, University of Alberta, Edmonton, Alberta, Canada.
- Alberta Machine Intelligence Institute, Edmonton, Alberta, Canada.
- Reeder AI, New York, New York, USA.
ORCID profiles: Cai Long, Deepak Subburam, Kayle Lowe, Jessica Zhang
Cite
APA
Long, C., Subburam, D., Lowe, K., Dos Santos, A., Zhang, J., Hwang, S., Saduka, N., Horev, Y., Su, T., Côté, D. W. J., & Wright, E. D. (2024). ChatENT: Augmented Large Language Model for Expert Knowledge Retrieval in Otolaryngology–Head and Neck Surgery. Otolaryngology–Head and Neck Surgery, 171(4), 1042-1051. https://doi.org/10.1002/ohn.864
Vancouver
Long C, Subburam D, Lowe K, Dos Santos A, Zhang J, Hwang S, et al. ChatENT: Augmented Large Language Model for Expert Knowledge Retrieval in Otolaryngology–Head and Neck Surgery. Otolaryngology–Head and Neck Surgery. 2024;171(4):1042-1051. doi:10.1002/ohn.864
BibTeX
@article{Long2024,
author = {Long, Cai and Subburam, Deepak and Lowe, Kayle and Dos Santos, André and Zhang, Jessica and Hwang, Sang and Saduka, Neil and Horev, Yoav and Su, Tao and Côté, David W J and Wright, Erin D},
title = {ChatENT: Augmented Large Language Model for Expert Knowledge Retrieval in Otolaryngology–Head and Neck Surgery},
journal = {Otolaryngology–Head and Neck Surgery},
year = {2024},
volume = {171},
number = {4},
pages = {1042--1051},
doi = {10.1002/ohn.864}
}
EUVIMED