A Survey on Temporal Sentence Grounding in Videos
ACM Transactions on Multimedia Computing, Communications, and Applications · 31.05.2023 · 19 (2) · 1-33
DOI: 10.1145/3532626 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 06.02.2023. Verfasst von Xiaohan Lan und 4 weiteren Autor:innen. Die Arbeit wurde bisher 40-mal zitiert (Crossref) und nennt 142 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China.
Abstract (Originalsprache)
Temporal sentence grounding in videos (TSGV), which aims at localizing one target segment from an untrimmed video with respect to a given sentence query, has drawn increasing attentions in the research community over the past few years. Different from the task of temporal action localization, TSGV is more flexible since it can locate complicated activities via natural languages, without restrictions from predefined action categories. Meanwhile, TSGV is more challenging since it requires both textual and visual understanding for semantic alignment between two modalities (i.e., text and video). In this survey, we give a comprehensive overview for TSGV, which (i) summarizes the taxonomy of existing methods, (ii) provides a detailed description of the evaluation protocols (i.e., datasets and metrics) to be used in TSGV, and (iii) in-depth discusses potential problems of current benchmarking designs and research directions for further investigations. To the best of our knowledge, this is the first systematic survey on temporal sentence grounding. More specifically, we first discuss existing TSGV approaches by grouping them into four categories, i.e., two-stage methods, single-stage methods, reinforcement learning-based methods, and weakly supervised methods. Then we present the benchmark datasets and evaluation metrics to assess current research progress. Finally, we discuss some limitations in TSGV through pointing out potential problems improperly resolved in the current evaluation protocols, which may push forwards more cutting-edge research in TSGV. Besides, we also share our insights on several promising directions, including four typical tasks with new and practical settings based on TSGV.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
Zitieren
APA
Lan, X., Yuan, Y., Wang, X., Wang, Z., & Zhu, W. (2023). A Survey on Temporal Sentence Grounding in Videos. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(2), 1-33. https://doi.org/10.1145/3532626
Vancouver
Lan X, Yuan Y, Wang X, Wang Z, Zhu W. A Survey on Temporal Sentence Grounding in Videos. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(2):1-33. doi:10.1145/3532626
BibTeX
@article{Lan2023,
author = {Lan, Xiaohan and Yuan, Yitian and Wang, Xin and Wang, Zhi and Zhu, Wenwu},
title = {A Survey on Temporal Sentence Grounding in Videos},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2023},
volume = {19},
number = {2},
pages = {1--33},
doi = {10.1145/3532626}
}
EUVIMED