Insights into the inner workings of transformer models for protein function prediction
Bioinformatics · 2024
DOI: 10.1093/bioinformatics/btae031 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioinformatics, 2024. Verfasst von Markus Wenzel und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Artificial Intelligence, Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin, Germ…. Hauptthemen laut MEDLINE: Artificial Intelligence, Amino Acids. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 112 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Bundesministerium für Bildung und Forschung, BIFOLD-Berlin Institute for the Foundations of Learning and Data, BIFOLD—Berlin Institute for the Foundations of Learning and Data.
Abstract (Originalsprache)
Motivation We explored how explainable artificial intelligence (XAI) can help to shed light into the inner workings of neural networks for protein function prediction, by extending the widely used XAI method of integrated gradients such that latent representations inside of transformer models, which were finetuned to Gene Ontology term and Enzyme Commission number prediction, can be inspected too. Results The approach enabled us to identify amino acids in the sequences that the transformers pay particular attention to, and to show that these relevant sequence parts reflect expectations from biology and chemistry, both in the embedding layer and inside of the model, where we identified transformer heads with a statistically significant correspondence of attribution maps with ground truth sequence annotations (e.g. transmembrane regions, active sites) across many proteins. Availability and Implementation Source code can be accessed at https://github.com/markuswenzel/xai-proteins.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Substanzen
Autor:innen und Institutionen
- Department of Artificial Intelligence, Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin, Germany.
- School VI - Medicine and Health Services, Carl von Ossietzky University of Oldenburg, Ammerländer Heerstr. 114-118, 26129 Oldenburg, Germany.
ORCID-Profile: Markus Wenzel, Nils Strodthoff
Förderung
- Bundesministerium für Bildung und Forschung
- BIFOLD-Berlin Institute for the Foundations of Learning and Data
- BIFOLD—Berlin Institute for the Foundations of Learning and Data
Zitieren
APA
Wenzel, M., Grüner, E., & Strodthoff, N. (2024). Insights into the inner workings of transformer models for protein function prediction. Bioinformatics. https://doi.org/10.1093/bioinformatics/btae031
Vancouver
Wenzel M, Grüner E, Strodthoff N. Insights into the inner workings of transformer models for protein function prediction. Bioinformatics. 2024. doi:10.1093/bioinformatics/btae031
BibTeX
@article{Wenzel2024,
author = {Wenzel, Markus and Grüner, Erik and Strodthoff, Nils},
title = {Insights into the inner workings of transformer models for protein function prediction},
journal = {Bioinformatics},
year = {2024},
doi = {10.1093/bioinformatics/btae031}
}
EUVIMED