EUVIMEDEUVIMED entdecken
ÜbersichtsarbeitOpen Access · CC BY 4.0

Applications of transformer-based language models in bioinformatics: a survey

Shuang Zhang iD, Rui Fan, Yuti Liu iD, Shuang Chen iD, Qiao Liu iD, Wanwen Zeng

Bioinformatics Advances · 05.01.2023 · 3 (1)

DOI: 10.1093/bioadv/vbad001 ↗

Kurz gefasst

Übersichtsarbeit in der Zeitschrift Bioinformatics Advances, online erschienen am 11.01.2023. Verfasst von Shuang Zhang und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Software, Nankai University, Tianjin 300350, China. Die Arbeit wurde bisher 186-mal zitiert (Crossref) und nennt 167 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, National Key Research and Development Program of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Summary The transformer-based language models, including vanilla transformer, BERT and GPT-3, have achieved revolutionary breakthroughs in the field of natural language processing (NLP). Since there are inherent similarities between various biological sequences and natural languages, the remarkable interpretability and adaptability of these models have prompted a new wave of their application in bioinformatics research. To provide a timely and comprehensive review, we introduce key developments of transformer-based language models by describing the detailed structure of transformers and summarize their contribution to a wide range of bioinformatics research from basic sequence analysis to drug discovery. While transformer-based applications in bioinformatics are diverse and multifaceted, we identify and discuss the common challenges, including heterogeneity of training data, computational expense and model interpretability, and opportunities in the context of bioinformatics research. We hope that the broader community of NLP researchers, bioinformaticians and biologists will be brought together to foster future research and development in transformer-based language models, and inspire novel bioinformatics applications that are unattainable by traditional methods. Supplementary information Supplementary data are available at Bioinformatics Advances online.

Autor:innen und Institutionen

  • College of Software, Nankai University, Tianjin 300350, China.
  • Department of Statistics, Stanford University, Stanford, CA 94305, USA.

ORCID-Profile: Shuang Zhang, Yuti Liu, Shuang Chen, Qiao Liu

Förderung

  • National Natural Science Foundation of China
  • National Key Research and Development Program of China

Zitieren

APA

Zhang, S., Fan, R., Liu, Y., Chen, S., Liu, Q., & Zeng, W. (2023). Applications of transformer-based language models in bioinformatics: a survey. Bioinformatics Advances, 3(1). https://doi.org/10.1093/bioadv/vbad001

Vancouver

Zhang S, Fan R, Liu Y, Chen S, Liu Q, Zeng W. Applications of transformer-based language models in bioinformatics: a survey. Bioinformatics Advances. 2023;3(1). doi:10.1093/bioadv/vbad001

BibTeX
@article{Zhang2023,
  author = {Zhang, Shuang and Fan, Rui and Liu, Yuti and Chen, Shuang and Liu, Qiao and Zeng, Wanwen},
  title = {Applications of transformer-based language models in bioinformatics: a survey},
  journal = {Bioinformatics Advances},
  year = {2023},
  volume = {3},
  number = {1},
  doi = {10.1093/bioadv/vbad001}
}
Kostenlos starten →