Efficient personalized image memorability via gaze-guided semantic cloning distillation
Knowledge and Information Systems · 2026-12 · 68 (1)
DOI: 10.1007/s10115-026-02835-w ↗Kurz gefasst
Fachartikel in der Zeitschrift Knowledge and Information Systems, online erschienen am 07.07.2026. Verfasst von Indrajeet Ghosh und 3 weiteren Autor:innen. Die Arbeit nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, Office of Naval Research, DEVCOM Army Research Laboratory.
Abstract (Originalsprache)
Image memorability (IM) estimation typically relies on learning generic semantic features from large-scale datasets; however, memorability is intrinsically individual-dependent and personalized visual viewing behavior, but overlooking this variability can undermine model performance in downstream cognitive and human–machine interaction tasks, leading toward suboptimal performance. To address this, we propose PerMem , a unified framework that integrates knowledge distillation (KD) and imitation learning (IL) to jointly learn generic and personalized salient representations for memorability estimation by leveraging both image content and gaze-derived heatmaps. PerMem employs a teacher network built upon a pre-trained ResNet-50 backbone, followed by an encoder–decoder architecture coupled with spatial and channel attention mechanisms to generate generic saliency-aware memorability maps. A lightweight, attention-guided student encoder–decoder is then optimized through a composite imitation-guided distillation process, where knowledge is distilled from the teacher while simultaneously imitating user-specific gaze fixation heatmaps. Through this joint training process, the student network learns to produce personalized memorability estimates while achieving substantial reductions in computational complexity. We validate PerMem on two public IM estimation datasets (LaMem and SUN) and a in-house WoM dataset comprising of 45 participants (UMBC IRB #670) engaged in visual search and navigation tasks that reflect individualized visual attention patterns. PerMem outperforms nine state-of-the-art IM models by capturing coarse-to-fine saliency and adapting to individual attention, achieving $$\approx $$ ≈ 6% improvement in memorability prediction. Lastly, we evaluate PerMem on heterogeneous embedded edge devices, including Jetson Nano, Jetson Xavier NX, and Raspberry Pi, demonstrating efficient on-device inference with consistent reductions in memory usage (25.6–33.4%), power consumption (19.0–25.0%), and inference latency (34.9–39.8%) relative to the teacher model, highlighting its practical robustness for resource-constrained, human-centric applications.
Förderung
- National Science Foundation
- Office of Naval Research
- DEVCOM Army Research Laboratory
Zitieren
APA
Ghosh, I., Anwar, M. S., Jayarajah, K., & Roy, N. (2026). Efficient personalized image memorability via gaze-guided semantic cloning distillation. Knowledge and Information Systems, 68(1). https://doi.org/10.1007/s10115-026-02835-w
Vancouver
Ghosh I, Anwar MS, Jayarajah K, Roy N. Efficient personalized image memorability via gaze-guided semantic cloning distillation. Knowledge and Information Systems. 2026;68(1). doi:10.1007/s10115-026-02835-w
BibTeX
@article{Ghosh2026,
author = {Ghosh, Indrajeet and Anwar, Mohammad Saeid and Jayarajah, Kasthuri and Roy, Nirmalya},
title = {Efficient personalized image memorability via gaze-guided semantic cloning distillation},
journal = {Knowledge and Information Systems},
year = {2026},
volume = {68},
number = {1},
doi = {10.1007/s10115-026-02835-w}
}
EUVIMED