EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Hierarchical Prototype Alignment for Video Temporal Grounding

Yun Tian iD, Xiaobo Guo iD, Jinsong Wang, Yuming Zhao, Bin Li iD

Entropy · 01.04.2026 · 28 (4) · 389

DOI: 10.3390/e28040389 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Entropy, online erschienen am 01.04.2026. Verfasst von Yun Tian und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China. Die Arbeit nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Jiangsu Engineering Research Center of Key Technology for Intelligent Manufacturing Equipment.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advances in vision-language cross-modal learning have substantially improved the performance of video temporal grounding. However, most existing methods directly associate global video features with sentence-level features, overlooking the fact that textual semantics usually correspond to only limited spatio-temporal regions within a video. This limitation often leads to unstable alignment in complex scenarios involving intertwined events and diverse actions. In essence, accurate video temporal grounding requires the joint modeling of fine-grained spatial semantics and heterogeneous temporal event structures. Motivated by this observation, we propose a hierarchical prototype alignment approach that models cross-modal correspondence between video and text through structured intermediate prototype representations. Specifically, the alignment process is decomposed into two complementary stages: object-phrase alignment and event-sentence alignment. In the object-phrase alignment stage, discriminative local visual regions and informative textual words are aggregated to construct object and phrase prototypes, thereby enhancing fine-grained spatial correspondence at the level of entities and localized actions. In the event-sentence alignment stage, object prototypes are further integrated along the temporal dimension to form event prototypes that represent continuous action units, enabling effective alignment with sentence-level semantics and facilitating the modeling of diverse temporal event structures. On this basis, we further directly inject cross-modal alignment information into candidate moment aggregation. This design allows candidate moment representations to emphasize query-relevant temporal regions. Extensive experiments on Charades-STA, ActivityNet Captions, and TACoS demonstrate that the proposed method outperforms existing approaches, validating the effectiveness of hierarchical prototype alignment for improving both cross-modal alignment quality and temporal grounding accuracy.

Schlagwörter

Stichwörter der Autor:innen

cross-modal alignmentprototype learningvideo temporal grounding

Autor:innen und Institutionen

  • School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China.
  • School of Mechanical and Electrical Engineering, Suqian University, Suqian 223800, China.
  • Jiangsu Engineering Research Center of Key Technology for Intelligent Manufacturing Equipment, Suqian 223800, China.
  • Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China.

ORCID-Profile: Yun Tian, Xiaobo Guo, Bin Li

Förderung

  • Jiangsu Engineering Research Center of Key Technology for Intelligent Manufacturing Equipment

Zitieren

APA

Tian, Y., Guo, X., Wang, J., Zhao, Y., & Li, B. (2026). Hierarchical Prototype Alignment for Video Temporal Grounding. Entropy, 28(4), 389. https://doi.org/10.3390/e28040389

Vancouver

Tian Y, Guo X, Wang J, Zhao Y, Li B. Hierarchical Prototype Alignment for Video Temporal Grounding. Entropy. 2026;28(4):389. doi:10.3390/e28040389

BibTeX
@article{Tian2026,
  author = {Tian, Yun and Guo, Xiaobo and Wang, Jinsong and Zhao, Yuming and Li, Bin},
  title = {Hierarchical Prototype Alignment for Video Temporal Grounding},
  journal = {Entropy},
  year = {2026},
  volume = {28},
  number = {4},
  pages = {389},
  doi = {10.3390/e28040389}
}
Kostenlos starten →