EUVIMEDEUVIMED entdecken
Scoping ReviewOpen Access

Task-Specific Transformer-Based Language Models in Health Care: Scoping Review

Ha Na Cho iD, Tae Joon Jun iD, Young-Hak Kim iD, Heejun Kang iD, Imjin Ahn iD, Hansle Gwon iD, Yunha Kim iD, Jiahn Seo iD, Heejung Choi iD, Minkyoung Kim iD, Jiye Han iD, Gaeun Kee iD und 2 weitere

JMIR Medical Informatics · 18.11.2024 · 12 · e49724

DOI: 10.2196/49724 ↗

Kurz gefasst

Scoping Review in der Zeitschrift JMIR Medical Informatics, online erschienen am 18.11.2024. Verfasst von Ha Na Cho und 13 weiteren Autor:innen. Beteiligte Einrichtung: Department of Information Medicine, Asan Medical Center, Seoul, Republic of Korea. Hauptthemen laut MEDLINE: Delivery of Health Care. Die Arbeit wurde bisher 80-mal zitiert (Crossref) und nennt 157 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Transformer-based language models have shown great potential to revolutionize health care by advancing clinical decision support, patient interaction, and disease prediction. However, despite their rapid development, the implementation of transformer-based language models in health care settings remains limited. This is partly due to the lack of a comprehensive review, which hinders a systematic understanding of their applications and limitations. Without clear guidelines and consolidated information, both researchers and physicians face difficulties in using these models effectively, resulting in inefficient research efforts and slow integration into clinical workflows.

Objective

This scoping review addresses this gap by examining studies on medical transformer-based language models and categorizing them into 6 tasks: dialogue generation, question answering, summarization, text classification, sentiment analysis, and named entity recognition.

Methods

We conducted a scoping review following the Cochrane scoping review protocol. A comprehensive literature search was performed across databases, including Google Scholar and PubMed, covering publications from January 2017 to September 2024. Studies involving transformer-derived models in medical tasks were included. Data were categorized into 6 key tasks.

Results

Our key findings revealed both advancements and critical challenges in applying transformer-based models to health care tasks. For example, models like MedPIR involving dialogue generation show promise but face privacy and ethical concerns, while question-answering models like BioBERT improve accuracy but struggle with the complexity of medical terminology. The BioBERTSum summarization model aids clinicians by condensing medical texts but needs better handling of long sequences.

Conclusions

This review attempted to provide a consolidated understanding of the role of transformer-based language models in health care and to guide future research directions. By addressing current challenges and exploring the potential for real-world applications, we envision significant improvements in health care informatics. Addressing the identified challenges and implementing proposed solutions can enable transformer-based language models to significantly improve health care delivery and patient outcomes. Our review provides valuable insights for future research and practical applications, setting the stage for transformative advancements in medical informatics.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Delivery of Health CareHumansNatural Language Processing

Stichwörter der Autor:innen

health caremedical language modelmedicinetransformer-based language models

Autor:innen und Institutionen

  • Department of Information Medicine, Asan Medical Center, Seoul, Republic of Korea.
  • Big Data Research Center, Asan Institute for Life Sciences, Asan Medical Center, Seoul, Republic of Korea.
  • Division of Cardiology, Department of Information Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
  • Division of Cardiology, Asan Medical Center, Seoul, Republic of Korea.
  • Department of Medical Science, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.

ORCID-Profile: Ha Na Cho, Tae Joon Jun, Young-Hak Kim, Heejun Kang, Imjin Ahn, Hansle Gwon, Yunha Kim, Jiahn Seo, Heejung Choi, Minkyoung Kim, Jiye Han, Gaeun Kee, Seohyun Park, Soyoung Ko

Zitieren

APA

Cho, H. N., Jun, T. J., Kim, Y. H., Kang, H., Ahn, I., Gwon, H., Kim, Y., Seo, J., Choi, H., Kim, M., Han, J., Kee, G., Park, S., & Ko, S. (2024). Task-Specific Transformer-Based Language Models in Health Care: Scoping Review. JMIR Medical Informatics, 12, e49724. https://doi.org/10.2196/49724

Vancouver

Cho HN, Jun TJ, Kim YH, Kang H, Ahn I, Gwon H, et al. Task-Specific Transformer-Based Language Models in Health Care: Scoping Review. JMIR Medical Informatics. 2024;12:e49724. doi:10.2196/49724

BibTeX
@article{Cho2024,
  author = {Cho, Ha Na and Jun, Tae Joon and Kim, Young-Hak and Kang, Heejun and Ahn, Imjin and Gwon, Hansle and Kim, Yunha and Seo, Jiahn and Choi, Heejung and Kim, Minkyoung and Han, Jiye and Kee, Gaeun and Park, Seohyun and Ko, Soyoung},
  title = {Task-Specific Transformer-Based Language Models in Health Care: Scoping Review},
  journal = {JMIR Medical Informatics},
  year = {2024},
  volume = {12},
  pages = {e49724},
  doi = {10.2196/49724}
}
Kostenlos starten →