EUVIMEDEUVIMED entdecken
Crossref

Phrase-Level Temporal Relationship Mining for Temporal Sentence Localization

Minghang Zheng, Sizhe Li, Qingchao Chen, Yuxin Peng, Yang Liu

Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (3) · 3669-3677

DOI: 10.1609/aaai.v37i3.25478 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Minghang Zheng und 4 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this paper, we address the problem of video temporal sentence localization, which aims to localize a target moment from videos according to a given language query. We observe that existing models suffer from a sheer performance drop when dealing with simple phrases contained in the sentence. It reveals the limitation that existing models only capture the annotation bias of the datasets but lack sufficient understanding of the semantic phrases in the query. To address this problem, we propose a phrase-level Temporal Relationship Mining (TRM) framework employing the temporal relationship relevant to the phrase and the whole sentence to have a better understanding of each semantic entity in the sentence. Specifically, we use phrase-level predictions to refine the sentence-level prediction, and use Multiple Instance Learning to improve the quality of phrase-level predictions. We also exploit the consistency and exclusiveness constraints of phrase-level and sentence-level predictions to regularize the training process, thus alleviating the ambiguity of each phrase prediction. The proposed approach sheds light on how machines can understand detailed phrases in a sentence and their compositions in their generality rather than learning the annotation biases. Experiments on the ActivityNet Captions and Charades-STA datasets show the effectiveness of our method on both phrase and sentence temporal localization and enable better model interpretability and generalization when dealing with unseen compositions of seen concepts. Code can be found at https://github.com/minghangz/TRM.

Zitieren

APA

Zheng, M., Li, S., Chen, Q., Peng, Y., & Liu, Y. (2023). Phrase-Level Temporal Relationship Mining for Temporal Sentence Localization. Proceedings of the AAAI Conference on Artificial Intelligence, 37(3), 3669-3677. https://doi.org/10.1609/aaai.v37i3.25478

Vancouver

Zheng M, Li S, Chen Q, Peng Y, Liu Y. Phrase-Level Temporal Relationship Mining for Temporal Sentence Localization. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(3):3669-3677. doi:10.1609/aaai.v37i3.25478

BibTeX
@article{Zheng2023,
  author = {Zheng, Minghang and Li, Sizhe and Chen, Qingchao and Peng, Yuxin and Liu, Yang},
  title = {Phrase-Level Temporal Relationship Mining for Temporal Sentence Localization},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2023},
  volume = {37},
  number = {3},
  pages = {3669--3677},
  doi = {10.1609/aaai.v37i3.25478}
}
Kostenlos starten →