Quantifying interpretation reproducibility in Vision Transformer models with TAVAC
Science Advances · 20.12.2024 · 10 (51)
DOI: 10.1126/sciadv.abg0264 ↗Kurz gefasst
Fachartikel in der Zeitschrift Science Advances, erschienen am 20.12.2024. Verfasst von Yue Zhao und 4 weiteren Autor:innen. Beteiligte Einrichtung: The Jackson Laboratory for Genomic Medicine, Farmington, CT, USA. Hauptthemen laut MEDLINE: Deep Learning. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIGMS NIH HHS, NIA NIH HHS, NCI NIH HHS.
Abstract (Originalsprache)
Deep learning algorithms can extract meaningful diagnostic features from biomedical images, promising improved patient care in digital pathology. Vision Transformer (ViT) models capture long-range spatial relationships and offer robust prediction power and better interpretability for image classification tasks than convolutional neural network models. However, limited annotated biomedical imaging datasets can cause ViT models to overfit, leading to false predictions due to random noise. To address this, we introduce Training Attention and Validation Attention Consistency (TAVAC), a metric for evaluating ViT model overfitting and quantifying interpretation reproducibility. By comparing high-attention regions between training and testing, we tested TAVAC on four public image classification datasets and two independent breast cancer histological image datasets. Overfitted models showed significantly lower TAVAC scores. TAVAC also distinguishes off-target from on-target attentions and measures interpretation generalization at a fine-grained cellular level. Beyond diagnostics, TAVAC enhances interpretative reproducibility in basic research, revealing critical spatial patterns and cellular structures of biomedical and other general nonbiomedical images.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- The Jackson Laboratory for Genomic Medicine, Farmington, CT, USA.
- Department of Mathematics and Statistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
- The Jackson Laboratory for Mouse Genetics, Bar Harbor, ME, USA.
- The Jackson Laboratory Cancer Center, Bar Harbor, ME, USA.
- Department of Biochemistry and Molecular Medicine, Department of Translational Genomics, Norris Comprehensive Cancer Center, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
ORCID-Profile: Yue Zhao, Dylan Agyemang, Yang Liu, Matt Mahoney, Sheng Li
Förderung
- NIGMS NIH HHS
- NIA NIH HHS
- NCI NIH HHS
- NHGRI NIH HHS
Zitieren
APA
Zhao, Y., Agyemang, D., Liu, Y., Mahoney, M., & Li, S. (2024). Quantifying interpretation reproducibility in Vision Transformer models with TAVAC. Science Advances, 10(51). https://doi.org/10.1126/sciadv.abg0264
Vancouver
Zhao Y, Agyemang D, Liu Y, Mahoney M, Li S. Quantifying interpretation reproducibility in Vision Transformer models with TAVAC. Science Advances. 2024;10(51). doi:10.1126/sciadv.abg0264
BibTeX
@article{Zhao2024,
author = {Zhao, Yue and Agyemang, Dylan and Liu, Yang and Mahoney, Matt and Li, Sheng},
title = {Quantifying interpretation reproducibility in Vision Transformer models with TAVAC},
journal = {Science Advances},
year = {2024},
volume = {10},
number = {51},
doi = {10.1126/sciadv.abg0264}
}
EUVIMED