EUVIMEDEUVIMED entdecken
Open Access

Quantifying interpretation reproducibility in Vision Transformer models with TAVAC

Yue Zhao iD, Dylan Agyemang iD, Yang Liu iD, Matt Mahoney iD, Sheng Li iD

Science Advances · 20.12.2024 · 10 (51)

DOI: 10.1126/sciadv.abg0264 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Science Advances, erschienen am 20.12.2024. Verfasst von Yue Zhao und 4 weiteren Autor:innen. Beteiligte Einrichtung: The Jackson Laboratory for Genomic Medicine, Farmington, CT, USA. Hauptthemen laut MEDLINE: Deep Learning. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIGMS NIH HHS, NIA NIH HHS, NCI NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep learning algorithms can extract meaningful diagnostic features from biomedical images, promising improved patient care in digital pathology. Vision Transformer (ViT) models capture long-range spatial relationships and offer robust prediction power and better interpretability for image classification tasks than convolutional neural network models. However, limited annotated biomedical imaging datasets can cause ViT models to overfit, leading to false predictions due to random noise. To address this, we introduce Training Attention and Validation Attention Consistency (TAVAC), a metric for evaluating ViT model overfitting and quantifying interpretation reproducibility. By comparing high-attention regions between training and testing, we tested TAVAC on four public image classification datasets and two independent breast cancer histological image datasets. Overfitted models showed significantly lower TAVAC scores. TAVAC also distinguishes off-target from on-target attentions and measures interpretation generalization at a fine-grained cellular level. Beyond diagnostics, TAVAC enhances interpretative reproducibility in basic research, revealing critical spatial patterns and cellular structures of biomedical and other general nonbiomedical images.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Deep LearningHumansReproducibility of ResultsAlgorithmsBreast NeoplasmsImage Processing, Computer-AssistedFemaleNeural Networks, ComputerImage Interpretation, Computer-Assisted

Autor:innen und Institutionen

  • The Jackson Laboratory for Genomic Medicine, Farmington, CT, USA.
  • Department of Mathematics and Statistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
  • The Jackson Laboratory for Mouse Genetics, Bar Harbor, ME, USA.
  • The Jackson Laboratory Cancer Center, Bar Harbor, ME, USA.
  • Department of Biochemistry and Molecular Medicine, Department of Translational Genomics, Norris Comprehensive Cancer Center, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.

ORCID-Profile: Yue Zhao, Dylan Agyemang, Yang Liu, Matt Mahoney, Sheng Li

Förderung

  • NIGMS NIH HHS
  • NIA NIH HHS
  • NCI NIH HHS
  • NHGRI NIH HHS

Zitieren

APA

Zhao, Y., Agyemang, D., Liu, Y., Mahoney, M., & Li, S. (2024). Quantifying interpretation reproducibility in Vision Transformer models with TAVAC. Science Advances, 10(51). https://doi.org/10.1126/sciadv.abg0264

Vancouver

Zhao Y, Agyemang D, Liu Y, Mahoney M, Li S. Quantifying interpretation reproducibility in Vision Transformer models with TAVAC. Science Advances. 2024;10(51). doi:10.1126/sciadv.abg0264

BibTeX
@article{Zhao2024,
  author = {Zhao, Yue and Agyemang, Dylan and Liu, Yang and Mahoney, Matt and Li, Sheng},
  title = {Quantifying interpretation reproducibility in Vision Transformer models with TAVAC},
  journal = {Science Advances},
  year = {2024},
  volume = {10},
  number = {51},
  doi = {10.1126/sciadv.abg0264}
}
Kostenlos starten →