EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Insights into the inner workings of transformer models for protein function prediction

Markus Wenzel iD, Erik Grüner, Nils Strodthoff iD

Bioinformatics · 2024

DOI: 10.1093/bioinformatics/btae031 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics, 2024. Verfasst von Markus Wenzel und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Artificial Intelligence, Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin, Germ…. Hauptthemen laut MEDLINE: Artificial Intelligence, Amino Acids. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 112 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Bundesministerium für Bildung und Forschung, BIFOLD-Berlin Institute for the Foundations of Learning and Data, BIFOLD—Berlin Institute for the Foundations of Learning and Data.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation We explored how explainable artificial intelligence (XAI) can help to shed light into the inner workings of neural networks for protein function prediction, by extending the widely used XAI method of integrated gradients such that latent representations inside of transformer models, which were finetuned to Gene Ontology term and Enzyme Commission number prediction, can be inspected too. Results The approach enabled us to identify amino acids in the sequences that the transformers pay particular attention to, and to show that these relevant sequence parts reflect expectations from biology and chemistry, both in the embedding layer and inside of the model, where we identified transformer heads with a statistically significant correspondence of attribution maps with ground truth sequence annotations (e.g. transmembrane regions, active sites) across many proteins. Availability and Implementation Source code can be accessed at https://github.com/markuswenzel/xai-proteins.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceAmino AcidsGene OntologyNeural Networks, ComputerProtein Domains

Substanzen

Amino Acids

Autor:innen und Institutionen

  • Department of Artificial Intelligence, Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin, Germany.
  • School VI - Medicine and Health Services, Carl von Ossietzky University of Oldenburg, Ammerländer Heerstr. 114-118, 26129 Oldenburg, Germany.

ORCID-Profile: Markus Wenzel, Nils Strodthoff

Förderung

  • Bundesministerium für Bildung und Forschung
  • BIFOLD-Berlin Institute for the Foundations of Learning and Data
  • BIFOLD—Berlin Institute for the Foundations of Learning and Data

Zitieren

APA

Wenzel, M., Grüner, E., & Strodthoff, N. (2024). Insights into the inner workings of transformer models for protein function prediction. Bioinformatics. https://doi.org/10.1093/bioinformatics/btae031

Vancouver

Wenzel M, Grüner E, Strodthoff N. Insights into the inner workings of transformer models for protein function prediction. Bioinformatics. 2024. doi:10.1093/bioinformatics/btae031

BibTeX
@article{Wenzel2024,
  author = {Wenzel, Markus and Grüner, Erik and Strodthoff, Nils},
  title = {Insights into the inner workings of transformer models for protein function prediction},
  journal = {Bioinformatics},
  year = {2024},
  doi = {10.1093/bioinformatics/btae031}
}
Kostenlos starten →