EUVIMEDEUVIMED entdecken
Crossref

Pre-trained Language Models in Biomedical Domain: A Systematic Survey

Benyou Wang, Qianqian Xie, Jiahuan Pei, Zhihong Chen, Prayag Tiwari, Zhao Li, Jie Fu

ACM Computing Surveys · 31.03.2024 · 56 (3) · 1-52

DOI: 10.1145/3611651 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 05.10.2023. Verfasst von Benyou Wang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 160-mal zitiert (Crossref) und nennt 346 Literaturangaben. Gefördert von: Chinese Key-Area Research and Development Program of Guangdong Province, Shenzhen Science and Technology Program, Guangdong Provincial Key Laboratory of Big Data Computing, The Chinese University of Hong Kong, Shenzhen, Shenzhen Key Research Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Pre-trained language models (PLMs) have been the de facto paradigm for most natural language processing tasks. This also benefits the biomedical domain: researchers from informatics, medicine, and computer science communities propose various PLMs trained on biomedical datasets, e.g., biomedical text, electronic health records, protein, and DNA sequences for various biomedical tasks. However, the cross-discipline characteristics of biomedical PLMs hinder their spreading among communities; some existing works are isolated from each other without comprehensive comparison and discussions. It is nontrivial to make a survey that not only systematically reviews recent advances in biomedical PLMs and their applications but also standardizes terminology and benchmarks. This article summarizes the recent progress of pre-trained language models in the biomedical domain and their applications in downstream biomedical tasks. Particularly, we discuss the motivations of PLMs in the biomedical domain and introduce the key concepts of pre-trained language models. We then propose a taxonomy of existing biomedical PLMs that categorizes them from various perspectives systematically. Plus, their applications in biomedical downstream tasks are exhaustively discussed, respectively. Last, we illustrate various limitations and future trends, which aims to provide inspiration for the future research.

Förderung

  • Chinese Key-Area Research and Development Program of Guangdong Province
  • Shenzhen Science and Technology Program
  • Guangdong Provincial Key Laboratory of Big Data Computing, The Chinese University of Hong Kong, Shenzhen, Shenzhen Key Research Project
  • Shenzhen Doctoral Startup Funding

Zitieren

APA

Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2024). Pre-trained Language Models in Biomedical Domain: A Systematic Survey. ACM Computing Surveys, 56(3), 1-52. https://doi.org/10.1145/3611651

Vancouver

Wang B, Xie Q, Pei J, Chen Z, Tiwari P, Li Z, et al. Pre-trained Language Models in Biomedical Domain: A Systematic Survey. ACM Computing Surveys. 2024;56(3):1-52. doi:10.1145/3611651

BibTeX
@article{Wang2024,
  author = {Wang, Benyou and Xie, Qianqian and Pei, Jiahuan and Chen, Zhihong and Tiwari, Prayag and Li, Zhao and Fu, Jie},
  title = {Pre-trained Language Models in Biomedical Domain: A Systematic Survey},
  journal = {ACM Computing Surveys},
  year = {2024},
  volume = {56},
  number = {3},
  pages = {1--52},
  doi = {10.1145/3611651}
}
Kostenlos starten →