First-Person Video Domain Adaptation With Multi-Scene Cross-Site Datasets and Attention-Based Methods
IEEE Transactions on Circuits and Systems for Video Technology · 2023-12 · 33 (12) · 7774-7788
DOI: 10.1109/tcsvt.2023.3281671 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-12. Verfasst von Xianyuan Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 63 Literaturangaben. Gefördert von: China Scholarship Council.
Abstract (Originalsprache)
Unsupervised Domain Adaptation (UDA) can transfer knowledge from labeled source data to unlabeled target data of the same categories. However, UDA for first-person video action recognition is an under-explored problem, with a lack of benchmark datasets and limited consideration of first-person video characteristics. Existing benchmark datasets provide videos with a single activity scene, e.g. kitchen, and similar global video statistics. However, multiple activity scenes and different global video statistics are still essential for developing robust UDA networks for real-world applications. To this end, we first introduce two first-person video domain adaptation datasets: ADL-7 and GTEA_KITCHEN-6. To the best of our knowledge, they are the first to provide multi-scene and cross-site settings for UDA problem on first-person video action recognition, promoting diversity. They provide five more domains based on the original three from existing datasets, enriching data for this area. They are also compatible with existing datasets, ensuring scalability. First-person videos have unique challenges, i.e. actions tend to occur in hand-object interaction areas. Therefore, networks paying more attention to such areas can benefit common feature learning in UDA. Attention mechanisms can endow networks with the ability to allocate resources adaptively for the important parts of the inputs and fade out the rest. Hence, we introduce channel-temporal attention modules to capture the channel-wise and temporal-wise relationships and model their inter-dependencies important to this characteristic. Moreover, we propose a Channel-Temporal Attention Network (CTAN) to integrate these modules into existing architectures. CTAN outperforms baselines on the new datasets and one existing dataset, EPIC-8.
Förderung
- China Scholarship Council
Zitieren
APA
Liu, X., Zhou, S., Lei, T., Jiang, P., Chen, Z., & Lu, H. (2023). First-Person Video Domain Adaptation With Multi-Scene Cross-Site Datasets and Attention-Based Methods. IEEE Transactions on Circuits and Systems for Video Technology, 33(12), 7774-7788. https://doi.org/10.1109/tcsvt.2023.3281671
Vancouver
Liu X, Zhou S, Lei T, Jiang P, Chen Z, Lu H. First-Person Video Domain Adaptation With Multi-Scene Cross-Site Datasets and Attention-Based Methods. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(12):7774-7788. doi:10.1109/tcsvt.2023.3281671
BibTeX
@article{Liu2023,
author = {Liu, Xianyuan and Zhou, Shuo and Lei, Tao and Jiang, Ping and Chen, Zhixiang and Lu, Haiping},
title = {First-Person Video Domain Adaptation With Multi-Scene Cross-Site Datasets and Attention-Based Methods},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2023},
volume = {33},
number = {12},
pages = {7774--7788},
doi = {10.1109/tcsvt.2023.3281671}
}
EUVIMED