EUVIMEDEUVIMED entdecken
Crossref

Improving Self-Supervised Medical Image Pre-Training by Early Alignment With Human Eye Gaze Information

Sheng Wang, Zihao Zhao, Zhenrong Shen, Bin Wang, Qian Wang, Dinggang Shen

IEEE Transactions on Medical Imaging · 2025

DOI: 10.1109/tmi.2025.3528965 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2025. Verfasst von Sheng Wang und 5 weiteren Autor:innen. Hauptthemen laut MEDLINE: Fixation, Ocular, Image Processing, Computer-Assisted, Supervised Machine Learning, Eye-Tracking Technology. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 40 Literaturangaben. Gefördert von: National Natural Science Foundation of China, China Ministry of Science and Technology, Shanghai Municipal Central Guided Local Science and Technology Development Fund.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Alignment between human knowledge and machine learning models is crucial for achieving efficient and interpretable AI systems. However, conventional self-supervised pre-training methods often suffer from low efficiency, as they do not incorporate human knowledge during the pre-training process and instead rely mainly on post-hoc alignment techniques. We propose Gaze Pre-Training (GzPT), a novel approach that introduces early alignment with human eye gaze information during the pre-training process to enhance both the learning efficiency and performance of self-supervised models. By leveraging contrastive learning to pull together images with similar gaze patterns, GzPT can effectively align the model with human attention during the pre-training. We demonstrate the effectiveness of our approach on three diverse medical image datasets, showing that GzPT can consistently outperform baseline methods and learn more meaningful and interpretable representations. Our findings also highlight the potential of incorporating human eye gaze as a form of passive knowledge to bridge the gap between human and machine learning in the self-supervised pre-training. Our code is available at Github.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Fixation, OcularImage Processing, Computer-AssistedSupervised Machine LearningEye-Tracking TechnologyHumansMachine LearningAlgorithmsDatabases, Factual

Förderung

  • National Natural Science Foundation of China
  • China Ministry of Science and Technology
  • Shanghai Municipal Central Guided Local Science and Technology Development Fund
  • Key Research and Development Program of Guangdong Province, China
  • High Performance Computing (HPC) Platform of ShanghaiTech University

Zitieren

APA

Wang, S., Zhao, Z., Shen, Z., Wang, B., Wang, Q., & Shen, D. (2025). Improving Self-Supervised Medical Image Pre-Training by Early Alignment With Human Eye Gaze Information. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/tmi.2025.3528965

Vancouver

Wang S, Zhao Z, Shen Z, Wang B, Wang Q, Shen D. Improving Self-Supervised Medical Image Pre-Training by Early Alignment With Human Eye Gaze Information. IEEE Transactions on Medical Imaging. 2025. doi:10.1109/tmi.2025.3528965

BibTeX
@article{Wang2025,
  author = {Wang, Sheng and Zhao, Zihao and Shen, Zhenrong and Wang, Bin and Wang, Qian and Shen, Dinggang},
  title = {Improving Self-Supervised Medical Image Pre-Training by Early Alignment With Human Eye Gaze Information},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2025},
  doi = {10.1109/tmi.2025.3528965}
}
Kostenlos starten →