Token-based fidelity scoring for trustworthy vision transformer interpretations in medical imaging
International Journal of Computer Assisted Radiology and Surgery · 04.05.2026 · 21 (5) · 1141-1149
DOI: 10.1007/s11548-026-03676-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, online erschienen am 04.05.2026. Verfasst von Utku Ozbulak und 3 weiteren Autor:innen. Beteiligte Einrichtung: Center for Biosystems and Biotech Data Science, Ghent University Global Campus, Incheon, Republic of Korea. utku.ozbulak@ghent.ac.kr. Hauptthemen laut MEDLINE: Diagnostic Imaging, Image Interpretation, Computer-Assisted. Die Arbeit nennt 35 Literaturangaben.
Abstract (Originalsprache)
PURPOSE: This work addresses the problem that interpretability methods for medical imaging models are still mostly evaluated by agreement with human annotations, even though such agreement does not necessarily indicate that an explanation is faithful to the model's actual decision-making. The goal is to quantify fidelity, defined here as the extent to which an explanation reflects the evidence that truly drives model predictions, independent of human intuition, bias, or incomplete annotation. METHODS: We propose a prediction confidence-based approach to estimate token importance in vision transformers and introduce the confidence-based fidelity score (CFS), which compares token importance signals with interpretability maps to measure how faithfully an explanation matches model-intrinsic reasoning. We evaluate the approach across three medical imaging datasets using vision transformers initialized with both supervised and self-supervised (SSL) pretraining. RESULTS: Across datasets, models, and explanation methods, the experiments reveal clear discrepancies between fidelity and human annotation alignment, indicating that explanations can appear plausible or overlap with annotated regions while still failing to reflect confidence-sensitive model evidence. CONCLUSION: The proposed token importance formulation and CFS provide a practical, model-aligned way to assess interpretability fidelity in transformer-based medical imaging, complementing annotation-based evaluation and supporting more reliable auditing and selection of explanation methods for clinical AI workflows.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Center for Biosystems and Biotech Data Science, Ghent University Global Campus, Incheon, Republic of Korea. utku.ozbulak@ghent.ac.kr.
- IDLab, ELIS, Ghent University, Ghent, Belgium. utku.ozbulak@ghent.ac.kr.
- Computational Data Sciences Department, George Mason University Korea, Incheon, Republic of Korea. utku.ozbulak@ghent.ac.kr.
- Center for Biosystems and Biotech Data Science, Ghent University Global Campus, Incheon, Republic of Korea.
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Ghent, Belgium.
- IDLab, ELIS, Ghent University, Ghent, Belgium.
Zitieren
APA
Ozbulak, U., Kang, S., De Neve, W., & Vankerschaver, J. (2026). Token-based fidelity scoring for trustworthy vision transformer interpretations in medical imaging. International Journal of Computer Assisted Radiology and Surgery, 21(5), 1141-1149. https://doi.org/10.1007/s11548-026-03676-2
Vancouver
Ozbulak U, Kang S, De Neve W, Vankerschaver J. Token-based fidelity scoring for trustworthy vision transformer interpretations in medical imaging. International Journal of Computer Assisted Radiology and Surgery. 2026;21(5):1141-1149. doi:10.1007/s11548-026-03676-2
BibTeX
@article{Ozbulak2026,
author = {Ozbulak, Utku and Kang, Solha and De Neve, Wesley and Vankerschaver, Joris},
title = {Token-based fidelity scoring for trustworthy vision transformer interpretations in medical imaging},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2026},
volume = {21},
number = {5},
pages = {1141--1149},
doi = {10.1007/s11548-026-03676-2}
}
EUVIMED