SASA-CLIP: Structure-Aware Alignment with a Gaussian Prior for Fine-Grained Video Action Recognition
Sensors · 02.08.2026 · 26 (15) · 4865
DOI: 10.3390/s26154865 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 02.08.2026. Verfasst von Xiaowei Han und 5 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer and Information Engineering, Harbin University of Commerce, Harbin 150028, China. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Heilongjiang Provincial Natural Science Foundation of China.
Abstract (Originalsprache)
Fine-grained video action recognition remains challenging because action categories often differ only in subtle inter-class variations and complex temporal dynamics. Recent Contrastive Language–Image Pre-training (CLIP)-based extensions perform well on general action recognition, but they typically rely on early global pooling of video features. Such coarse representations discard the fine temporal cues that distinguish subtle actions, causing a granularity mismatch in cross-modal alignment. To address this, we propose Structure-Aware Semantic-Adaptive (SASA)-CLIP, a framework for multi-granular cross-modal alignment. SASA-CLIP adopts a dual-branch design: a coarse-grained branch captures the global context, while a fine-grained branch matches descriptors against individual frames before aggregation, rather than pooling features early. To keep this alignment temporally coherent, we introduce a Gaussian prior as a temporal structural constraint, encoding the inductive bias of local temporal continuity into the attention matrix to guide an ordered alignment of key action segments along the temporal axis. On Kinetics-400 (ViT-B/32), SASA-CLIP reaches a Top-1 accuracy of 81.37%, improving over the X-CLIP baseline by 0.97%; on HMDB-51 and UCF-101 (ViT-B/16), it reaches 74.0% and 96.81%, improving by 3.25% and 2.61%, respectively. It also transfers to the zero-shot setting, improving over the baseline on HMDB-51 and UCF-101. These results show that combining multi-granular representations with a temporal structural prior benefits fine-grained recognition, suggesting that SASA-CLIP is a practical option for real-world visual sensing applications such as intelligent surveillance and wearable activity monitoring.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computer and Information Engineering, Harbin University of Commerce, Harbin 150028, China.
- Heilongjiang Provincial Key Laboratory of Electronic Commerce and Information Processing, Harbin 150028, China.
Förderung
- Heilongjiang Provincial Natural Science Foundation of China
Zitieren
APA
Han, X., Si, W., Zhang, H., Yang, M., Ma, L., & Feng, Y. (2026). SASA-CLIP: Structure-Aware Alignment with a Gaussian Prior for Fine-Grained Video Action Recognition. Sensors, 26(15), 4865. https://doi.org/10.3390/s26154865
Vancouver
Han X, Si W, Zhang H, Yang M, Ma L, Feng Y. SASA-CLIP: Structure-Aware Alignment with a Gaussian Prior for Fine-Grained Video Action Recognition. Sensors. 2026;26(15):4865. doi:10.3390/s26154865
BibTeX
@article{Han2026,
author = {Han, Xiaowei and Si, Wenbao and Zhang, Honghui and Yang, Maolin and Ma, Lin and Feng, Yibo},
title = {SASA-CLIP: Structure-Aware Alignment with a Gaussian Prior for Fine-Grained Video Action Recognition},
journal = {Sensors},
year = {2026},
volume = {26},
number = {15},
pages = {4865},
doi = {10.3390/s26154865}
}
EUVIMED