EUVIMEDEUVIMED entdecken
Crossref

CoSTA: End-to-End Comprehensive Space-Time Entanglement for Spatio-Temporal Video Grounding

Yaoyuan Liang, Xiao Liang, Yansong Tang, Zhao Yang, Ziran Li, Jingang Wang, Wenbo Ding, Shao-Lun Huang

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (4) · 3324-3332

DOI: 10.1609/aaai.v38i4.28118 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yaoyuan Liang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper studies the spatio-temporal video grounding task, which aims to localize a spatio-temporal tube in an untrimmed video based on the given text description of an event. Existing one-stage approaches suffer from insufficient space-time interaction in two aspects: i) less precise prediction of event temporal boundaries, and ii) inconsistency in object prediction for the same event across adjacent frames. To address these issues, we propose a framework of Comprehensive Space-Time entAnglement (CoSTA) to densely entangle space-time multi-modal features for spatio-temporal localization. Specifically, we propose a space-time collaborative encoder to extract comprehensive video features and leverage Transformer to perform spatio-temporal multi-modal understanding. Our entangled decoder couples temporal boundary prediction and spatial localization via an entangled query, boasting an enhanced ability to capture object-event relationships. We conduct extensive experiments on the challenging benchmarks of HC-STVG and VidSTG, where CoSTA outperforms existing state-of-the-art methods, demonstrating its effectiveness for this task.

Zitieren

APA

Liang, Y., Liang, X., Tang, Y., Yang, Z., Li, Z., Wang, J., Ding, W., & Huang, S. (2024). CoSTA: End-to-End Comprehensive Space-Time Entanglement for Spatio-Temporal Video Grounding. Proceedings of the AAAI Conference on Artificial Intelligence, 38(4), 3324-3332. https://doi.org/10.1609/aaai.v38i4.28118

Vancouver

Liang Y, Liang X, Tang Y, Yang Z, Li Z, Wang J, et al. CoSTA: End-to-End Comprehensive Space-Time Entanglement for Spatio-Temporal Video Grounding. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(4):3324-3332. doi:10.1609/aaai.v38i4.28118

BibTeX
@article{Liang2024,
  author = {Liang, Yaoyuan and Liang, Xiao and Tang, Yansong and Yang, Zhao and Li, Ziran and Wang, Jingang and Ding, Wenbo and Huang, Shao-Lun},
  title = {CoSTA: End-to-End Comprehensive Space-Time Entanglement for Spatio-Temporal Video Grounding},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {4},
  pages = {3324--3332},
  doi = {10.1609/aaai.v38i4.28118}
}
Kostenlos starten →