EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Survey on Temporal Sentence Grounding in Videos

Xiaohan Lan, Yitian Yuan, Xin Wang, Zhi Wang, Wenwu Zhu

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.05.2023 · 19 (2) · 1-33

DOI: 10.1145/3532626 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 06.02.2023. Verfasst von Xiaohan Lan und 4 weiteren Autor:innen. Die Arbeit wurde bisher 40-mal zitiert (Crossref) und nennt 142 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Temporal sentence grounding in videos (TSGV), which aims at localizing one target segment from an untrimmed video with respect to a given sentence query, has drawn increasing attentions in the research community over the past few years. Different from the task of temporal action localization, TSGV is more flexible since it can locate complicated activities via natural languages, without restrictions from predefined action categories. Meanwhile, TSGV is more challenging since it requires both textual and visual understanding for semantic alignment between two modalities (i.e., text and video). In this survey, we give a comprehensive overview for TSGV, which (i) summarizes the taxonomy of existing methods, (ii) provides a detailed description of the evaluation protocols (i.e., datasets and metrics) to be used in TSGV, and (iii) in-depth discusses potential problems of current benchmarking designs and research directions for further investigations. To the best of our knowledge, this is the first systematic survey on temporal sentence grounding. More specifically, we first discuss existing TSGV approaches by grouping them into four categories, i.e., two-stage methods, single-stage methods, reinforcement learning-based methods, and weakly supervised methods. Then we present the benchmark datasets and evaluation metrics to assess current research progress. Finally, we discuss some limitations in TSGV through pointing out potential problems improperly resolved in the current evaluation protocols, which may push forwards more cutting-edge research in TSGV. Besides, we also share our insights on several promising directions, including four typical tasks with new and practical settings based on TSGV.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China

Zitieren

APA

Lan, X., Yuan, Y., Wang, X., Wang, Z., & Zhu, W. (2023). A Survey on Temporal Sentence Grounding in Videos. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(2), 1-33. https://doi.org/10.1145/3532626

Vancouver

Lan X, Yuan Y, Wang X, Wang Z, Zhu W. A Survey on Temporal Sentence Grounding in Videos. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(2):1-33. doi:10.1145/3532626

BibTeX
@article{Lan2023,
  author = {Lan, Xiaohan and Yuan, Yitian and Wang, Xin and Wang, Zhi and Zhu, Wenwu},
  title = {A Survey on Temporal Sentence Grounding in Videos},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2023},
  volume = {19},
  number = {2},
  pages = {1--33},
  doi = {10.1145/3532626}
}
Kostenlos starten →