EUVIMEDEUVIMED entdecken
Crossref

TubeRMC: Tube-conditioned Reconstruction with Mutual Constraints for Weakly-supervised Spatio-Temporal Video Grounding

Jinxuan Li, Yi Zhang, Jian-Fang Hu, Chaolei Tan, Tianming Liang, Beihao Xia

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (8) · 6253-6261

DOI: 10.1609/aaai.v40i8.37551 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Jinxuan Li und 5 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Spatio-Temporal Video Grounding (STVG) aims to localize a spatio-temporal tube that corresponds to a given language query in an untrimmed video. This is a challenging task since it involves complex vision-language understanding and spatiotemporal reasoning. Recent works have explored weakly-supervised setting in STVG to eliminate reliance on fine-grained annotations like bounding boxes or temporal stamps. However, they typically follow a simple late-fusion manner, which generates tubes independent of the text description, often resulting in failed target identification and inconsistent target tracking. To address this limitation, we propose a Tube-conditioned Reconstruction with Mutual Constraints (TubeRMC) framework that generates text-conditioned candidate tubes with pre-trained visual grounding models and further refine them via tube-conditioned reconstruction with spatio-temporal constraints. Specifically, we design three reconstruction strategies from temporal, spatial, and spatio-temporal perspectives to comprehensively capture rich tube-text correspondences. Each strategy is equipped with a Tube-conditioned Reconstructor, utilizing spatio-temporal tubes as condition to reconstruct the key clues in the query. We further introduce mutual constraints between spatial and temporal proposals to enhance their quality for reconstruction. TubeRMC outperforms existing methods on two public benchmarks VidSTG and HCSTVG. Further visualization shows that TubeRMC effectively mitigates both target identification errors and inconsistent tracking.

Zitieren

APA

Li, J., Zhang, Y., Hu, J., Tan, C., Liang, T., & Xia, B. (2026). TubeRMC: Tube-conditioned Reconstruction with Mutual Constraints for Weakly-supervised Spatio-Temporal Video Grounding. Proceedings of the AAAI Conference on Artificial Intelligence, 40(8), 6253-6261. https://doi.org/10.1609/aaai.v40i8.37551

Vancouver

Li J, Zhang Y, Hu J, Tan C, Liang T, Xia B. TubeRMC: Tube-conditioned Reconstruction with Mutual Constraints for Weakly-supervised Spatio-Temporal Video Grounding. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(8):6253-6261. doi:10.1609/aaai.v40i8.37551

BibTeX
@article{Li2026,
  author = {Li, Jinxuan and Zhang, Yi and Hu, Jian-Fang and Tan, Chaolei and Liang, Tianming and Xia, Beihao},
  title = {TubeRMC: Tube-conditioned Reconstruction with Mutual Constraints for Weakly-supervised Spatio-Temporal Video Grounding},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {8},
  pages = {6253--6261},
  doi = {10.1609/aaai.v40i8.37551}
}
Kostenlos starten →