A Snippets Relation and Hard-Snippets Mask Network for Weakly-Supervised Temporal Action Localization
IEEE Transactions on Circuits and Systems for Video Technology · 2024-08 · 34 (8) · 7202-7215
DOI: 10.1109/tcsvt.2024.3374870 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-08. Verfasst von Yibo Zhao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 71 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Tianjin Natural Science Foundation, Young Creative Team in Universities of Shandong Province.
Abstract (Originalsprache)
Weakly-supervised temporal action localization (WTAL) is a problem learning an action localization model with only video-level labels available. In recent years, many WTAL methods have developed. However, hard-to-predict snippets near action boundaries are often not considered in these existing approaches, causing action incompleteness and action over-complete issues. To solve these issues, in this work, an end-to-end snippets relation and hard-snippets mask network (SRHN) is proposed. Specifically, a hard-snippets mask module is applied to mask the hard-to-predict snippets adaptively, and in this way, the trained model focuses more on those snippets with low uncertainty. Then, a snippets relation module is designed to capture the relationship among snippets and can make hard-to-predict snippets easy to predict by aggregating the information of multiple temporal receptive fields. Finally, a snippet enhancement loss is further developed to reduce the action probabilities that are not present in videos for hard-to-predict snippets and other snippets, enlarging the action probabilities that exist in videos. Extensive experiments on THUMOS14, ActivityNet1.2, and ActivityNet1.3 datasets demonstrate the effectiveness of the SRHN method.
Förderung
- National Natural Science Foundation of China
- Tianjin Natural Science Foundation
- Young Creative Team in Universities of Shandong Province
- Jinan 20 Projects in Universities
- Shandong Project towards the Integration of Education and Industry
Zitieren
APA
Zhao, Y., Zhang, H., Gao, Z., Guan, W., Wang, M., & Chen, S. (2024). A Snippets Relation and Hard-Snippets Mask Network for Weakly-Supervised Temporal Action Localization. IEEE Transactions on Circuits and Systems for Video Technology, 34(8), 7202-7215. https://doi.org/10.1109/tcsvt.2024.3374870
Vancouver
Zhao Y, Zhang H, Gao Z, Guan W, Wang M, Chen S. A Snippets Relation and Hard-Snippets Mask Network for Weakly-Supervised Temporal Action Localization. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(8):7202-7215. doi:10.1109/tcsvt.2024.3374870
BibTeX
@article{Zhao2024,
author = {Zhao, Yibo and Zhang, Hua and Gao, Zan and Guan, Weili and Wang, Meng and Chen, Shengyong},
title = {A Snippets Relation and Hard-Snippets Mask Network for Weakly-Supervised Temporal Action Localization},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2024},
volume = {34},
number = {8},
pages = {7202--7215},
doi = {10.1109/tcsvt.2024.3374870}
}
EUVIMED