Improving Self-Supervised Medical Image Pre-Training by Early Alignment With Human Eye Gaze Information
IEEE Transactions on Medical Imaging · 2025
DOI: 10.1109/tmi.2025.3528965 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2025. Verfasst von Sheng Wang und 5 weiteren Autor:innen. Hauptthemen laut MEDLINE: Fixation, Ocular, Image Processing, Computer-Assisted, Supervised Machine Learning, Eye-Tracking Technology. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 40 Literaturangaben. Gefördert von: National Natural Science Foundation of China, China Ministry of Science and Technology, Shanghai Municipal Central Guided Local Science and Technology Development Fund.
Abstract (Originalsprache)
Alignment between human knowledge and machine learning models is crucial for achieving efficient and interpretable AI systems. However, conventional self-supervised pre-training methods often suffer from low efficiency, as they do not incorporate human knowledge during the pre-training process and instead rely mainly on post-hoc alignment techniques. We propose Gaze Pre-Training (GzPT), a novel approach that introduces early alignment with human eye gaze information during the pre-training process to enhance both the learning efficiency and performance of self-supervised models. By leveraging contrastive learning to pull together images with similar gaze patterns, GzPT can effectively align the model with human attention during the pre-training. We demonstrate the effectiveness of our approach on three diverse medical image datasets, showing that GzPT can consistently outperform baseline methods and learn more meaningful and interpretable representations. Our findings also highlight the potential of incorporating human eye gaze as a form of passive knowledge to bridge the gap between human and machine learning in the self-supervised pre-training. Our code is available at Github.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- National Natural Science Foundation of China
- China Ministry of Science and Technology
- Shanghai Municipal Central Guided Local Science and Technology Development Fund
- Key Research and Development Program of Guangdong Province, China
- High Performance Computing (HPC) Platform of ShanghaiTech University
Zitieren
APA
Wang, S., Zhao, Z., Shen, Z., Wang, B., Wang, Q., & Shen, D. (2025). Improving Self-Supervised Medical Image Pre-Training by Early Alignment With Human Eye Gaze Information. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/tmi.2025.3528965
Vancouver
Wang S, Zhao Z, Shen Z, Wang B, Wang Q, Shen D. Improving Self-Supervised Medical Image Pre-Training by Early Alignment With Human Eye Gaze Information. IEEE Transactions on Medical Imaging. 2025. doi:10.1109/tmi.2025.3528965
BibTeX
@article{Wang2025,
author = {Wang, Sheng and Zhao, Zihao and Shen, Zhenrong and Wang, Bin and Wang, Qian and Shen, Dinggang},
title = {Improving Self-Supervised Medical Image Pre-Training by Early Alignment With Human Eye Gaze Information},
journal = {IEEE Transactions on Medical Imaging},
year = {2025},
doi = {10.1109/tmi.2025.3528965}
}
EUVIMED