EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ProkBERT family: genomic language models for microbiome applications

Balázs Ligeti, István Szepesi-Nagy, Babett Bodnár, Noémi Ligeti-Nagy, János Juhász

Frontiers in Microbiology · 12.01.2024 · 14

DOI: 10.3389/fmicb.2023.1331233 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Microbiology, online erschienen am 12.01.2024. Verfasst von Balázs Ligeti und 4 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Budapest, Hungary. Die Arbeit wurde bisher 36-mal zitiert (Crossref) und nennt 83 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: HSRD VA.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

In the evolving landscape of microbiology and microbiome analysis, the integration of machine learning is crucial for understanding complex microbial interactions, and predicting and recognizing novel functionalities within extensive datasets. However, the effectiveness of these methods in microbiology faces challenges due to the complex and heterogeneous nature of microbial data, further complicated by low signal-to-noise ratios, context-dependency, and a significant shortage of appropriately labeled datasets. This study introduces the ProkBERT model family, a collection of large language models, designed for genomic tasks. It provides a generalizable sequence representation for nucleotide sequences, learned from unlabeled genome data. This approach helps overcome the above-mentioned limitations in the field, thereby improving our understanding of microbial ecosystems and their impact on health and disease.

Methods

ProkBERT models are based on transfer learning and self-supervised methodologies, enabling them to use the abundant yet complex microbial data effectively. The introduction of the novel Local Context-Aware (LCA) tokenization technique marks a significant advancement, allowing ProkBERT to overcome the contextual limitations of traditional transformer models. This methodology not only retains rich local context but also demonstrates remarkable adaptability across various bioinformatics tasks.

Results

In practical applications such as promoter prediction and phage identification, the ProkBERT models show superior performance. For promoter prediction tasks, the top-performing model achieved a Matthews Correlation Coefficient (MCC) of 0.74 for E. coli and 0.62 in mixed-species contexts. In phage identification, ProkBERT models consistently outperformed established tools like VirSorter2 and DeepVirFinder, achieving an MCC of 0.85. These results underscore the models' exceptional accuracy and generalizability in both supervised and unsupervised tasks.

Conclusions

The ProkBERT model family is a compact yet powerful tool in the field of microbiology and bioinformatics. Its capacity for rapid, accurate analyses and its adaptability across a spectrum of tasks marks a significant advancement in machine learning applications in microbiology. The models are available on GitHub ( https://github.com/nbrg-ppcu/prokbert ) and HuggingFace ( https://huggingface.co/nerualbioinfo ) providing an accessible tool for the community.

Schlagwörter

Stichwörter der Autor:innen

BERTLCA tokenizationgenomic language modelslanguage modelsmachine learning in microbiologyphagepromotertransformer models

Autor:innen und Institutionen

  • Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Budapest, Hungary.
  • Language Technology Research Group, HUN-REN Hungarian Research Centre for Linguistics, Budapest, Hungary.
  • Institute of Medical Microbiology, Semmelweis University, Budapest, Hungary.

Förderung

  • HSRD VA

Zitieren

APA

Ligeti, B., Szepesi-Nagy, I., Bodnár, B., Ligeti-Nagy, N., & Juhász, J. (2024). ProkBERT family: genomic language models for microbiome applications. Frontiers in Microbiology, 14. https://doi.org/10.3389/fmicb.2023.1331233

Vancouver

Ligeti B, Szepesi-Nagy I, Bodnár B, Ligeti-Nagy N, Juhász J. ProkBERT family: genomic language models for microbiome applications. Frontiers in Microbiology. 2024;14. doi:10.3389/fmicb.2023.1331233

BibTeX
@article{Ligeti2024,
  author = {Ligeti, Balázs and Szepesi-Nagy, István and Bodnár, Babett and Ligeti-Nagy, Noémi and Juhász, János},
  title = {ProkBERT family: genomic language models for microbiome applications},
  journal = {Frontiers in Microbiology},
  year = {2024},
  volume = {14},
  doi = {10.3389/fmicb.2023.1331233}
}
Kostenlos starten →