Knowledge Diffusion-Based Adaptive Alignment With Hierarchical Context for Video Temporal Grounding
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-10 · 48 (10) · 11901-11915
DOI: 10.1109/tpami.2026.3694223 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-10. Verfasst von Ran Ran und 6 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 82 Literaturangaben. Gefördert von: Fundamental and Interdisciplinary Disciplines Breakthrough Plan, Ministry of Education of China, National Natural Science Foundation of China.
Abstract (Originalsprache)
Video Temporal Grounding (VTG) aims to localize the temporal moment that corresponds to a natural-language query in an untrimmed video. However, VTG faces two core challenges: narrowing the semantic gap between concise text queries and complex visual content, and extracting discriminative moment cues from long background-dominant sequences. To address these challenges, we propose Knowledge Diffusion Alignment (KDA), a diffusion-based framework that operates in the semantic feature domain. At its core, a Background Residual Diffusion Model (BRDM) starts from adaptive global video knowledge and, under query conditioning, progressively removes query-irrelevant background residuals to synthesize moment knowledge. In addition, to provide a strong conditional prior and multi-scale supervision, Hierarchical Video Knowledge Extraction (HVKE) aggregates context at multiple temporal granularities, producing both video and moment knowledge. KDA is trained in three stages: first learning target-aware moment knowledge, then aligning the diffusion process to that target, and finally performing end-to-end inference using BRDM-generated knowledge. The synthesized moment knowledge is injected into knowledge prompt reasoning, where a Dynamic Moment-aware Encoder (DME) adaptively generates dynamic temporal convolution kernels conditioned on the moment knowledge, coupling efficient local modeling with global self-attention. For prediction, a Spans-Enhanced Hierarchical Decoder (SEHD) constructs a temporal pyramid and adaptively selects the most relevant initial spans to exploit intrinsic alignment cues. Experiments across public benchmarks demonstrate that the proposed KDA outperforms the state-of-the-art methods, and extensive ablations further verify the effectiveness of the module design.
Förderung
- Fundamental and Interdisciplinary Disciplines Breakthrough Plan
- Ministry of Education of China
- National Natural Science Foundation of China
Zitieren
APA
Ran, R., Wei, J., Yang, Y., Wang, G., Liu, K., Zou, J., & Shen, H. T. (2026). Knowledge Diffusion-Based Adaptive Alignment With Hierarchical Context for Video Temporal Grounding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(10), 11901-11915. https://doi.org/10.1109/tpami.2026.3694223
Vancouver
Ran R, Wei J, Yang Y, Wang G, Liu K, Zou J, et al. Knowledge Diffusion-Based Adaptive Alignment With Hierarchical Context for Video Temporal Grounding. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(10):11901-11915. doi:10.1109/tpami.2026.3694223
BibTeX
@article{Ran2026,
author = {Ran, Ran and Wei, Jiwei and Yang, Yang and Wang, Guoqing and Liu, Kuien and Zou, Jie and Shen, Heng Tao},
title = {Knowledge Diffusion-Based Adaptive Alignment With Hierarchical Context for Video Temporal Grounding},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {10},
pages = {11901--11915},
doi = {10.1109/tpami.2026.3694223}
}
EUVIMED