EUVIMEDEUVIMED entdecken
Crossref

Action-Driven Semantic Representation and Aggregation for Video Captioning

Tingting Han, Yaochen Xu, Jun Yu, Zhou Yu, Sicheng Zhao

IEEE Transactions on Circuits and Systems for Video Technology · 2025-04 · 35 (4) · 3383-3395

DOI: 10.1109/tcsvt.2024.3502736 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-04. Verfasst von Tingting Han und 4 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 51 Literaturangaben. Gefördert von: Zhejiang Province Natural Science Foundation, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video captioning, a challenging task that entails generating natural language descriptions of visual content, often fails to effectively grasp the essence of action semantics. To harness the power of action detection to facilitate a deeper understanding of the video content, we propose an action-driven method, named Hierarchical Semantic Representation and Aggregation (HSRA) network. This method explicitly exploits action clues with a hierarchical semantic representation module, which models visual semantics in a three-level structure: “object-action-event”. By employing learnable action queries, our approach injects extensive action semantics into the model, thereby enabling more accurate and context-rich captions. To further enhance semantic alignment and understanding, we introduce a semantic aggregation composed of a semantic interaction module and a semantic refinement module. This component facilitates the alignment of semantics across different levels and emphasizes key information, ultimately leading to significant improvements in semantic consistency between the video and generated captions. We performed extensive evaluations on two well-established public datasets, MSVD and MSR-VTT, and the findings consistently demonstrate that our proposed HSRA network outperforms contemporary state-of-the-art methods.

Förderung

  • Zhejiang Province Natural Science Foundation
  • National Natural Science Foundation of China

Zitieren

APA

Han, T., Xu, Y., Yu, J., Yu, Z., & Zhao, S. (2025). Action-Driven Semantic Representation and Aggregation for Video Captioning. IEEE Transactions on Circuits and Systems for Video Technology, 35(4), 3383-3395. https://doi.org/10.1109/tcsvt.2024.3502736

Vancouver

Han T, Xu Y, Yu J, Yu Z, Zhao S. Action-Driven Semantic Representation and Aggregation for Video Captioning. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(4):3383-3395. doi:10.1109/tcsvt.2024.3502736

BibTeX
@article{Han2025,
  author = {Han, Tingting and Xu, Yaochen and Yu, Jun and Yu, Zhou and Zhao, Sicheng},
  title = {Action-Driven Semantic Representation and Aggregation for Video Captioning},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {4},
  pages = {3383--3395},
  doi = {10.1109/tcsvt.2024.3502736}
}
Kostenlos starten →