EUVIMEDEUVIMED entdecken
Crossref

First-Person Video Domain Adaptation With Multi-Scene Cross-Site Datasets and Attention-Based Methods

Xianyuan Liu, Shuo Zhou, Tao Lei, Ping Jiang, Zhixiang Chen, Haiping Lu

IEEE Transactions on Circuits and Systems for Video Technology · 2023-12 · 33 (12) · 7774-7788

DOI: 10.1109/tcsvt.2023.3281671 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-12. Verfasst von Xianyuan Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 63 Literaturangaben. Gefördert von: China Scholarship Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Unsupervised Domain Adaptation (UDA) can transfer knowledge from labeled source data to unlabeled target data of the same categories. However, UDA for first-person video action recognition is an under-explored problem, with a lack of benchmark datasets and limited consideration of first-person video characteristics. Existing benchmark datasets provide videos with a single activity scene, e.g. kitchen, and similar global video statistics. However, multiple activity scenes and different global video statistics are still essential for developing robust UDA networks for real-world applications. To this end, we first introduce two first-person video domain adaptation datasets: ADL-7 and GTEA_KITCHEN-6. To the best of our knowledge, they are the first to provide multi-scene and cross-site settings for UDA problem on first-person video action recognition, promoting diversity. They provide five more domains based on the original three from existing datasets, enriching data for this area. They are also compatible with existing datasets, ensuring scalability. First-person videos have unique challenges, i.e. actions tend to occur in hand-object interaction areas. Therefore, networks paying more attention to such areas can benefit common feature learning in UDA. Attention mechanisms can endow networks with the ability to allocate resources adaptively for the important parts of the inputs and fade out the rest. Hence, we introduce channel-temporal attention modules to capture the channel-wise and temporal-wise relationships and model their inter-dependencies important to this characteristic. Moreover, we propose a Channel-Temporal Attention Network (CTAN) to integrate these modules into existing architectures. CTAN outperforms baselines on the new datasets and one existing dataset, EPIC-8.

Förderung

  • China Scholarship Council

Zitieren

APA

Liu, X., Zhou, S., Lei, T., Jiang, P., Chen, Z., & Lu, H. (2023). First-Person Video Domain Adaptation With Multi-Scene Cross-Site Datasets and Attention-Based Methods. IEEE Transactions on Circuits and Systems for Video Technology, 33(12), 7774-7788. https://doi.org/10.1109/tcsvt.2023.3281671

Vancouver

Liu X, Zhou S, Lei T, Jiang P, Chen Z, Lu H. First-Person Video Domain Adaptation With Multi-Scene Cross-Site Datasets and Attention-Based Methods. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(12):7774-7788. doi:10.1109/tcsvt.2023.3281671

BibTeX
@article{Liu2023,
  author = {Liu, Xianyuan and Zhou, Shuo and Lei, Tao and Jiang, Ping and Chen, Zhixiang and Lu, Haiping},
  title = {First-Person Video Domain Adaptation With Multi-Scene Cross-Site Datasets and Attention-Based Methods},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2023},
  volume = {33},
  number = {12},
  pages = {7774--7788},
  doi = {10.1109/tcsvt.2023.3281671}
}
Kostenlos starten →