Progressive Localization Networks for Language-Based Moment Localization
ACM Transactions on Multimedia Computing, Communications, and Applications · 31.05.2023 · 19 (2) · 1-21
DOI: 10.1145/3543857 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 06.02.2023. Verfasst von Qi Zheng und 7 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 80 Literaturangaben. Gefördert von: National Key R&D Program of China, NSFC, Public Welfare Technology Research Project of Zhejiang Province.
Abstract (Originalsprache)
This article targets the task of language-based video moment localization. The language-based setting of this task allows for an open set of target activities, resulting in a large variation of the temporal lengths of video moments. Most existing methods prefer to first sample sufficient candidate moments with various temporal lengths, then match them with the given query to determine the target moment. However, candidate moments generated with a fixed temporal granularity may be suboptimal to handle the large variation in moment lengths. To this end, we propose a novel multi-stage Progressive Localization Network (PLN) that progressively localizes the target moment in a coarse-to-fine manner. Specifically, each stage of PLN has a localization branch and focuses on candidate moments that are generated with a specific temporal granularity. The temporal granularities of candidate moments are different across the stages. Moreover, we devise a conditional feature manipulation module and an upsampling connection to bridge the multiple localization branches. In this fashion, the later stages are able to absorb the previously learned information, thus facilitating the more fine-grained localization. Extensive experiments on three public datasets demonstrate the effectiveness of our proposed PLN for language-based moment localization, especially for localizing short moments in long videos.
Förderung
- National Key R&D Program of China
- NSFC
- Public Welfare Technology Research Project of Zhejiang Province
- Fundamental Research Funds for the Provincial Universities of Zhejiang
- Open Projects Program of the National Laboratory of Pattern Recognition
Zitieren
APA
Zheng, Q., Dong, J., Qu, X., Yang, X., Wang, Y., Zhou, P., Liu, B., & Wang, X. (2023). Progressive Localization Networks for Language-Based Moment Localization. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(2), 1-21. https://doi.org/10.1145/3543857
Vancouver
Zheng Q, Dong J, Qu X, Yang X, Wang Y, Zhou P, et al. Progressive Localization Networks for Language-Based Moment Localization. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(2):1-21. doi:10.1145/3543857
BibTeX
@article{Zheng2023,
author = {Zheng, Qi and Dong, Jianfeng and Qu, Xiaoye and Yang, Xun and Wang, Yabing and Zhou, Pan and Liu, Baolong and Wang, Xun},
title = {Progressive Localization Networks for Language-Based Moment Localization},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2023},
volume = {19},
number = {2},
pages = {1--21},
doi = {10.1145/3543857}
}
EUVIMED