EUVIMEDObjevte EUVIMED
Scoping ReviewOpen access

Task-Specific Transformer-Based Language Models in Health Care: Scoping Review

Ha Na Cho iD, Tae Joon Jun iD, Young-Hak Kim iD, Heejun Kang iD, Imjin Ahn iD, Hansle Gwon iD, Yunha Kim iD, Jiahn Seo iD, Heejung Choi iD, Minkyoung Kim iD, Jiye Han iD, Gaeun Kee iD and 2 more

JMIR Medical Informatics · 2024-11-18 · 12 · e49724

DOI: 10.2196/49724 ↗

At a glance

Scoping Review in JMIR Medical Informatics, published online on 2024-11-18. Written by Ha Na Cho and 13 co-authors. Institution: Department of Information Medicine, Asan Medical Center, Seoul, Republic of Korea. Major MEDLINE topics: Delivery of Health Care. Cited 80 times so far (Crossref); 157 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Transformer-based language models have shown great potential to revolutionize health care by advancing clinical decision support, patient interaction, and disease prediction. However, despite their rapid development, the implementation of transformer-based language models in health care settings remains limited. This is partly due to the lack of a comprehensive review, which hinders a systematic understanding of their applications and limitations. Without clear guidelines and consolidated information, both researchers and physicians face difficulties in using these models effectively, resulting in inefficient research efforts and slow integration into clinical workflows.

Objective

This scoping review addresses this gap by examining studies on medical transformer-based language models and categorizing them into 6 tasks: dialogue generation, question answering, summarization, text classification, sentiment analysis, and named entity recognition.

Methods

We conducted a scoping review following the Cochrane scoping review protocol. A comprehensive literature search was performed across databases, including Google Scholar and PubMed, covering publications from January 2017 to September 2024. Studies involving transformer-derived models in medical tasks were included. Data were categorized into 6 key tasks.

Results

Our key findings revealed both advancements and critical challenges in applying transformer-based models to health care tasks. For example, models like MedPIR involving dialogue generation show promise but face privacy and ethical concerns, while question-answering models like BioBERT improve accuracy but struggle with the complexity of medical terminology. The BioBERTSum summarization model aids clinicians by condensing medical texts but needs better handling of long sequences.

Conclusions

This review attempted to provide a consolidated understanding of the role of transformer-based language models in health care and to guide future research directions. By addressing current challenges and exploring the potential for real-world applications, we envision significant improvements in health care informatics. Addressing the identified challenges and implementing proposed solutions can enable transformer-based language models to significantly improve health care delivery and patient outcomes. Our review provides valuable insights for future research and practical applications, setting the stage for transformative advancements in medical informatics.

Subjects

MeSH terms (MEDLINE)

Delivery of Health CareHumansNatural Language Processing

Author keywords

health caremedical language modelmedicinetransformer-based language models

Authors and institutions

  • Department of Information Medicine, Asan Medical Center, Seoul, Republic of Korea.
  • Big Data Research Center, Asan Institute for Life Sciences, Asan Medical Center, Seoul, Republic of Korea.
  • Division of Cardiology, Department of Information Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
  • Division of Cardiology, Asan Medical Center, Seoul, Republic of Korea.
  • Department of Medical Science, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.

ORCID profiles: Ha Na Cho, Tae Joon Jun, Young-Hak Kim, Heejun Kang, Imjin Ahn, Hansle Gwon, Yunha Kim, Jiahn Seo, Heejung Choi, Minkyoung Kim, Jiye Han, Gaeun Kee, Seohyun Park, Soyoung Ko

Cite

APA

Cho, H. N., Jun, T. J., Kim, Y. H., Kang, H., Ahn, I., Gwon, H., Kim, Y., Seo, J., Choi, H., Kim, M., Han, J., Kee, G., Park, S., & Ko, S. (2024). Task-Specific Transformer-Based Language Models in Health Care: Scoping Review. JMIR Medical Informatics, 12, e49724. https://doi.org/10.2196/49724

Vancouver

Cho HN, Jun TJ, Kim YH, Kang H, Ahn I, Gwon H, et al. Task-Specific Transformer-Based Language Models in Health Care: Scoping Review. JMIR Medical Informatics. 2024;12:e49724. doi:10.2196/49724

BibTeX
@article{Cho2024,
  author = {Cho, Ha Na and Jun, Tae Joon and Kim, Young-Hak and Kang, Heejun and Ahn, Imjin and Gwon, Hansle and Kim, Yunha and Seo, Jiahn and Choi, Heejung and Kim, Minkyoung and Han, Jiye and Kee, Gaeun and Park, Seohyun and Ko, Soyoung},
  title = {Task-Specific Transformer-Based Language Models in Health Care: Scoping Review},
  journal = {JMIR Medical Informatics},
  year = {2024},
  volume = {12},
  pages = {e49724},
  doi = {10.2196/49724}
}
Start for free →