EUVIMEDEUVIMED entdecken
Crossref

Human-Centric Transformer for Domain Adaptive Action Recognition

Kun-Yu Lin, Jiaming Zhou, Wei-Shi Zheng

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-02 · 47 (2) · 679-696

DOI: 10.1109/tpami.2024.3429387 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-02. Verfasst von Kun-Yu Lin und 2 weiteren Autor:innen. Hauptthemen laut MEDLINE: Pattern Recognition, Automated, Image Processing, Computer-Assisted, Machine Learning. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 126 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Guangdong NSF Project.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We study the domain adaptation task for action recognition, namely domain adaptive action recognition, which aims to effectively transfer action recognition power from a label-sufficient source domain to a label-free target domain. Since actions are performed by humans, it is crucial to exploit human cues in videos when recognizing actions across domains. However, existing methods are prone to losing human cues but prefer to exploit the correlation between non-human contexts and associated actions for recognition, and the contexts of interest agnostic to actions would reduce recognition performance in the target domain. To overcome this problem, we focus on uncovering human-centric action cues for domain adaptive action recognition, and our conception is to investigate two aspects of human-centric action cues, namely human cues and human-context interaction cues. Accordingly, our proposed Human-Centric Transformer (HCTransformer) develops a decoupled human-centric learning paradigm to explicitly concentrate on human-centric action cues in domain-variant video feature learning. Our HCTransformer first conducts human-aware temporal modeling by a human encoder, aiming to avoid a loss of human cues during domain-invariant video feature learning. Then, by a Transformer-like architecture, HCTransformer exploits domain-invariant and action-correlated contexts by a context encoder, and further models domain-invariant interaction between humans and action-correlated contexts. We conduct extensive experiments on three benchmarks, namely UCF-HMDB, Kinetics-NecDrone and EPIC-Kitchens-UDA, and the state-of-the-art performance demonstrates the effectiveness of our proposed HCTransformer.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Pattern Recognition, AutomatedImage Processing, Computer-AssistedMachine LearningHumansVideo RecordingAlgorithmsCuesHuman Activities

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Guangdong NSF Project

Zitieren

APA

Lin, K. Y., Zhou, J., & Zheng, W. S. (2025). Human-Centric Transformer for Domain Adaptive Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(2), 679-696. https://doi.org/10.1109/tpami.2024.3429387

Vancouver

Lin KY, Zhou J, Zheng WS. Human-Centric Transformer for Domain Adaptive Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(2):679-696. doi:10.1109/tpami.2024.3429387

BibTeX
@article{Lin2025,
  author = {Lin, Kun-Yu and Zhou, Jiaming and Zheng, Wei-Shi},
  title = {Human-Centric Transformer for Domain Adaptive Action Recognition},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {2},
  pages = {679--696},
  doi = {10.1109/tpami.2024.3429387}
}
Kostenlos starten →