Human-Centric Transformer for Domain Adaptive Action Recognition
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-02 · 47 (2) · 679-696
DOI: 10.1109/tpami.2024.3429387 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-02. Verfasst von Kun-Yu Lin und 2 weiteren Autor:innen. Hauptthemen laut MEDLINE: Pattern Recognition, Automated, Image Processing, Computer-Assisted, Machine Learning. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 126 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Guangdong NSF Project.
Abstract (Originalsprache)
We study the domain adaptation task for action recognition, namely domain adaptive action recognition, which aims to effectively transfer action recognition power from a label-sufficient source domain to a label-free target domain. Since actions are performed by humans, it is crucial to exploit human cues in videos when recognizing actions across domains. However, existing methods are prone to losing human cues but prefer to exploit the correlation between non-human contexts and associated actions for recognition, and the contexts of interest agnostic to actions would reduce recognition performance in the target domain. To overcome this problem, we focus on uncovering human-centric action cues for domain adaptive action recognition, and our conception is to investigate two aspects of human-centric action cues, namely human cues and human-context interaction cues. Accordingly, our proposed Human-Centric Transformer (HCTransformer) develops a decoupled human-centric learning paradigm to explicitly concentrate on human-centric action cues in domain-variant video feature learning. Our HCTransformer first conducts human-aware temporal modeling by a human encoder, aiming to avoid a loss of human cues during domain-invariant video feature learning. Then, by a Transformer-like architecture, HCTransformer exploits domain-invariant and action-correlated contexts by a context encoder, and further models domain-invariant interaction between humans and action-correlated contexts. We conduct extensive experiments on three benchmarks, namely UCF-HMDB, Kinetics-NecDrone and EPIC-Kitchens-UDA, and the state-of-the-art performance demonstrates the effectiveness of our proposed HCTransformer.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Guangdong NSF Project
Zitieren
APA
Lin, K. Y., Zhou, J., & Zheng, W. S. (2025). Human-Centric Transformer for Domain Adaptive Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(2), 679-696. https://doi.org/10.1109/tpami.2024.3429387
Vancouver
Lin KY, Zhou J, Zheng WS. Human-Centric Transformer for Domain Adaptive Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(2):679-696. doi:10.1109/tpami.2024.3429387
BibTeX
@article{Lin2025,
author = {Lin, Kun-Yu and Zhou, Jiaming and Zheng, Wei-Shi},
title = {Human-Centric Transformer for Domain Adaptive Action Recognition},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
volume = {47},
number = {2},
pages = {679--696},
doi = {10.1109/tpami.2024.3429387}
}
EUVIMED