EUVIMEDEUVIMED entdecken
Crossref

Toward Open-Set Human Object Interaction Detection

Mingrui Wu, Yuqi Liu, Jiayi Ji, Xiaoshuai Sun, Rongrong Ji

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (6) · 6066-6073

DOI: 10.1609/aaai.v38i6.28422 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Mingrui Wu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This work is oriented toward the task of open-set Human Object Interaction (HOI) detection. The challenge lies in identifying completely new, out-of-domain relationships, as opposed to in-domain ones which have seen improvements in zero-shot HOI detection. To address this challenge, we introduce a simple Disentangled HOI Detection (DHD) model for detecting novel relationships by integrating an open-set object detector with a Visual Language Model (VLM). We utilize a disentangled image-text contrastive learning metric for training and connect the bottom-up visual features to text embeddings through lightweight unary and pair-wise adapters. Our model can benefit from the open-set object detector and the VLM to detect novel action categories and combine actions with novel object categories. We further present the VG-HOI dataset, a comprehensive benchmark with over 17k HOI relationships for open-set scenarios. Experimental results show that our model can detect unknown action classes and combine unknown object classes. Furthermore, it can generalize to over 17k HOI classes while being trained on just 600 HOI classes.

Zitieren

APA

Wu, M., Liu, Y., Ji, J., Sun, X., & Ji, R. (2024). Toward Open-Set Human Object Interaction Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 38(6), 6066-6073. https://doi.org/10.1609/aaai.v38i6.28422

Vancouver

Wu M, Liu Y, Ji J, Sun X, Ji R. Toward Open-Set Human Object Interaction Detection. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(6):6066-6073. doi:10.1609/aaai.v38i6.28422

BibTeX
@article{Wu2024,
  author = {Wu, Mingrui and Liu, Yuqi and Ji, Jiayi and Sun, Xiaoshuai and Ji, Rongrong},
  title = {Toward Open-Set Human Object Interaction Detection},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {6},
  pages = {6066--6073},
  doi = {10.1609/aaai.v38i6.28422}
}
Kostenlos starten →