EUVIMEDEUVIMED entdecken
Crossref

Large language models in ORL – Danger or opportunity?

Christoph Raphael Buhr, Harry Smith, Tilman Huppertz, Katharina Bahr-Hamm, Christoph Matthias, Clemens Cuny, Jan Phillipp Snijders, Benjamin Ernst, Andrew Blaikie, Tom Kelsey, Sebastian Kuhn, Jonas Eckrich

Laryngo-Rhino-Otologie · 01.05.2024 · 103 (S 02 S 02) · S181

DOI: 10.1055/s-0044-1784577 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Laryngo-Rhino-Otologie, erschienen am 01.05.2024. Verfasst von Christoph Raphael Buhr und 11 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The wide range of information available from Large Lange Models (LLMs) together with the semantically high-quality responses make LLMs powerful and consequently attractive as sources of medical information for laypersons and medical staff alike. There is however risk of generating misinformation. The aim of this study was to compare the quality and safety of two popular LLMs: ChatGPT4 and Bard. 41 case-based ENT questions were posed to the two LLMs (ChatGPT4 and Bard) as well as 6 ORL consultants. A blinded evaluation of the answers was carried out by the consultants which included their opinion on whether they were generated by a consultant or a LLM and if there was potential patient risk from misinformation. In addition coherence, comprehensibility, conciseness as well as the medical quality of the answers were rated on a 6-point Likert scale. Consultants did not evaluate their own answers. Consultants achieved better results than the LLMs among all evaluation criteria. In particular, the doctors scored better in the assessment of medical quality. A difference between the two LLMs was also noted. The LLMs scored highly for semantic quality, reflected by good results in the categories of coherence, comprehensibility and conciseness. On average however the LLMs used statistically more characters than the doctors. While LLM responses exhibited notably lower medical content quality compared to consultants, their high linguistic proficiency may suggests a higher degree of expertise. This could mislead users into perceiving the information as accurate, when it may contain misinformation posing a further potential risk for widespread use. Despite this limitation in offering medical advice, the overall caliber of the responses generated by LLMs remains good. Publication History Article published online: 19 April 2024 © 2024. Thieme. All rights reserved. Georg Thieme Verlag KG Rüdigerstraße 14, 70469 Stuttgart, Germany

Zitieren

APA

Buhr, C. R., Smith, H., Huppertz, T., Bahr-Hamm, K., Matthias, C., Cuny, C., Snijders, J. P., Ernst, B., Blaikie, A., Kelsey, T., Kuhn, S., & Eckrich, J. (2024). Large language models in ORL – Danger or opportunity?. Laryngo-Rhino-Otologie, 103(S 02 S 02), S181. https://doi.org/10.1055/s-0044-1784577

Vancouver

Buhr CR, Smith H, Huppertz T, Bahr-Hamm K, Matthias C, Cuny C, et al. Large language models in ORL – Danger or opportunity?. Laryngo-Rhino-Otologie. 2024;103(S 02 S 02):S181. doi:10.1055/s-0044-1784577

BibTeX
@article{Buhr2024,
  author = {Buhr, Christoph Raphael and Smith, Harry and Huppertz, Tilman and Bahr-Hamm, Katharina and Matthias, Christoph and Cuny, Clemens and Snijders, Jan Phillipp and Ernst, Benjamin and Blaikie, Andrew and Kelsey, Tom and Kuhn, Sebastian and Eckrich, Jonas},
  title = {Large language models in ORL – Danger or opportunity?},
  journal = {Laryngo-Rhino-Otologie},
  year = {2024},
  volume = {103},
  number = {S 02 S 02},
  pages = {S181},
  doi = {10.1055/s-0044-1784577}
}
Kostenlos starten →