EUVIMEDEUVIMED entdecken
Crossref

DeTAL: Open-Vocabulary Temporal Action Localization With Decoupled Networks

Zhiheng Li, Yujie Zhong, Ran Song, Tianjiao Li, Lin Ma, Wei Zhang

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 7728-7741

DOI: 10.1109/tpami.2024.3395778 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von Zhiheng Li und 5 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 71 Literaturangaben. Gefördert von: National Natural Science Foundation of China, National Key Research and Development Plan of China, Shandong Excellent Young Scientists Fund Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Pre-trained visual-language (ViL) models have demonstrated good zero-shot capability in video understanding tasks, where they were usually adapted through fine-tuning or temporal modeling. However, in the task of open-vocabulary temporal action localization (OV-TAL), such adaption reduces the robustness of ViL models against different data distributions, leading to a misalignment between visual representations and text descriptions of unseen action categories. As a result, existing methods often strike a trade-off between action detection and classification. Aiming at this issue, this paper proposes DeTAL, a simple but effective two-stage approach for OV-TAL. DeTAL decouples action detection from action classification to avoid the compromise between them, and the state-of-the-art methods for close-set action localization can be handily adapted to OV-TAL, which significantly improves the performance. Meanwhile, DeTAL can easily tackle the scenario where action category annotations are unavailable in the training dataset. In the experiments, we propose a new cross-dataset setting to evaluate the zero-shot capability of different methods. And the results demonstrate that DeTAL outperforms the state-of-the-art methods for OV-TAL on both THUMOS14 and ActivityNet1.3.

Förderung

  • National Natural Science Foundation of China
  • National Key Research and Development Plan of China
  • Shandong Excellent Young Scientists Fund Program
  • Self-Developed Innovation Team of Jinan City
  • Young Taishan Scholars Program of Shandong Province

Zitieren

APA

Li, Z., Zhong, Y., Song, R., Li, T., Ma, L., & Zhang, W. (2024). DeTAL: Open-Vocabulary Temporal Action Localization With Decoupled Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 7728-7741. https://doi.org/10.1109/tpami.2024.3395778

Vancouver

Li Z, Zhong Y, Song R, Li T, Ma L, Zhang W. DeTAL: Open-Vocabulary Temporal Action Localization With Decoupled Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):7728-7741. doi:10.1109/tpami.2024.3395778

BibTeX
@article{Li2024,
  author = {Li, Zhiheng and Zhong, Yujie and Song, Ran and Li, Tianjiao and Ma, Lin and Zhang, Wei},
  title = {DeTAL: Open-Vocabulary Temporal Action Localization With Decoupled Networks},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {12},
  pages = {7728--7741},
  doi = {10.1109/tpami.2024.3395778}
}
Kostenlos starten →