HR-Pro: Point-Supervised Temporal Action Localization via Hierarchical Reliability Propagation
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (7) · 7115-7123
DOI: 10.1609/aaai.v38i7.28539 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Huaxin Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 31-mal zitiert (Crossref).
Abstract (Originalsprache)
Point-supervised Temporal Action Localization (PSTAL) is an emerging research direction for label-efficient learning. However, current methods mainly focus on optimizing the network either at the snippet-level or the instance-level, neglecting the inherent reliability of point annotations at both levels. In this paper, we propose a Hierarchical Reliability Propagation (HR-Pro) framework, which consists of two reliability-aware stages: Snippet-level Discrimination Learning and Instance-level Completeness Learning, both stages explore the efficient propagation of high-confidence cues in point annotations. For snippet-level learning, we introduce an online-updated memory to store reliable snippet prototypes for each class. We then employ a Reliability-aware Attention Block to capture both intra-video and inter-video dependencies of snippets, resulting in more discriminative and robust snippet representation. For instance-level learning, we propose a point-based proposal generation approach as a means of connecting snippets and instances, which produces high-confidence proposals for further optimization at the instance level. Through multi-level reliability-aware learning, we obtain more reliable confidence scores and more accurate temporal boundaries of predicted proposals. Our HR-Pro achieves state-of-the-art performance on multiple challenging benchmarks, including an impressive average mAP of 60.3% on THUMOS14. Notably, our HR-Pro largely surpasses all previous point-supervised methods, and even outperforms several competitive fully-supervised methods. Code will be available at https://github.com/pipixin321/HR-Pro.
Zitieren
APA
Zhang, H., Wang, X., Xu, X., Qing, Z., Gao, C., & Sang, N. (2024). HR-Pro: Point-Supervised Temporal Action Localization via Hierarchical Reliability Propagation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(7), 7115-7123. https://doi.org/10.1609/aaai.v38i7.28539
Vancouver
Zhang H, Wang X, Xu X, Qing Z, Gao C, Sang N. HR-Pro: Point-Supervised Temporal Action Localization via Hierarchical Reliability Propagation. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(7):7115-7123. doi:10.1609/aaai.v38i7.28539
BibTeX
@article{Zhang2024,
author = {Zhang, Huaxin and Wang, Xiang and Xu, Xiaohao and Qing, Zhiwu and Gao, Changxin and Sang, Nong},
title = {HR-Pro: Point-Supervised Temporal Action Localization via Hierarchical Reliability Propagation},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {7},
pages = {7115--7123},
doi = {10.1609/aaai.v38i7.28539}
}
EUVIMED