EUVIMEDEUVIMED entdecken
Crossref

Spatio-Temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition

Hongyu Qu, Xiangbo Shu, Rui Yan, Hailiang Gao, Wenguan Wang, Jinhui Tang

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-07 · 48 (7) · 7954-7968

DOI: 10.1109/tpami.2026.3669254 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-07. Verfasst von Hongyu Qu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 31-mal zitiert (Crossref) und nennt 87 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Zhejiang Provincial Natural Science Foundation of China, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Few-Shot Action Recognition (FSAR) is a challenging task that requires recognizing novel action categories with a few labeled videos. Recent works typically apply semantically coarse category names as auxiliary contexts to guide the learning of discriminative visual features. However, such context provided by the action names is too limited to provide sufficient background knowledge for capturing novel spatial and temporal concepts in actions. In this paper, we propose DiST, an innovative Decomposition-incorporation framework for FSAR that makes use of decoupled Spatial and Temporal knowledge provided by large language models to learn expressive multi-granularity prototypes. In the decomposition stage, we decouple vanilla action names into diverse spatio-temporal attribute descriptions (action-related knowledge). Such commonsense knowledge complements semantic contexts from spatial and temporal perspectives. In the incorporation stage, we propose Spatial/Temporal Knowledge Compensators (SKC/TKC) to discover discriminative object-level and frame-level prototypes, respectively. In SKC, object-level prototypes adaptively aggregate important patch tokens under the guidance of spatial knowledge. Moreover, in TKC, frame-level prototypes utilize temporal attributes to assist in inter-frame temporal relation modeling. These learned prototypes thus provide transparency in capturing fine-grained spatial details and diverse temporal patterns. Experimental results show DiST achieves state-of-the-art results on five standard FSAR datasets.

Förderung

  • National Natural Science Foundation of China
  • Zhejiang Provincial Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities
  • Jiangsu Provincial Scientific Research Center of Applied Mathematics
  • Postgraduate Research & Practice Innovation Program of Jiangsu Province

Zitieren

APA

Qu, H., Shu, X., Yan, R., Gao, H., Wang, W., & Tang, J. (2026). Spatio-Temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(7), 7954-7968. https://doi.org/10.1109/tpami.2026.3669254

Vancouver

Qu H, Shu X, Yan R, Gao H, Wang W, Tang J. Spatio-Temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(7):7954-7968. doi:10.1109/tpami.2026.3669254

BibTeX
@article{Qu2026,
  author = {Qu, Hongyu and Shu, Xiangbo and Yan, Rui and Gao, Hailiang and Wang, Wenguan and Tang, Jinhui},
  title = {Spatio-Temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {7},
  pages = {7954--7968},
  doi = {10.1109/tpami.2026.3669254}
}
Kostenlos starten →