HOPE: A Hierarchical Perspective for Semi-Supervised 2D-3D Cross-Modal Retrieval
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 8976-8993
DOI: 10.1109/tpami.2024.3412760 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von Fan Zhang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 97 Literaturangaben.
Abstract (Originalsprache)
With the emergence of AI generated content, cross-modal retrieval of 2D and 3D data has obtained increasing research attention. In practical applications, massive amounts of 2D and 3D data need expensive annotation, which would make labels scarce. Even worse, complicated heterogeneous relationships between 2D and 3D data make the problem more challenging. In this research, we study the problem of semi-supervised 2D and 3D cross-modal retrieval and provide a novel method named Hierarchical Alignment with Ambiguous Pseudo-labeling (HOPE) for this problem. The core of HOPE is to align two modalities in the common space from a hierarchical perspective. Specifically, HOPE not only enforces each sample to approach its respective modality-invariant anchors from an individual view, but also measures both prototypes and distribution for both modalities for discrepancy reduction from a group view. To handle label scarcity with limited error accumulation, HOPE employs two branches of perturbed networks to generate ambiguous candidates, which guides the cross-branch supervision using a margin-based ranking objective. In addition, we retrieve reliable unlabeled samples for each anchor with curriculum learning and class balance, which are added into labeled datasets to clear ambiguity. Extensive experiments on various benchmark datasets validate the superiority of the proposed HOPE.
Zitieren
APA
Zhang, F., Zhou, H., Hua, X., Chen, C., & Luo, X. (2024). HOPE: A Hierarchical Perspective for Semi-Supervised 2D-3D Cross-Modal Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 8976-8993. https://doi.org/10.1109/tpami.2024.3412760
Vancouver
Zhang F, Zhou H, Hua X, Chen C, Luo X. HOPE: A Hierarchical Perspective for Semi-Supervised 2D-3D Cross-Modal Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):8976-8993. doi:10.1109/tpami.2024.3412760
BibTeX
@article{Zhang2024,
author = {Zhang, Fan and Zhou, Hang and Hua, Xian-Sheng and Chen, Chong and Luo, Xiao},
title = {HOPE: A Hierarchical Perspective for Semi-Supervised 2D-3D Cross-Modal Retrieval},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {12},
pages = {8976--8993},
doi = {10.1109/tpami.2024.3412760}
}
EUVIMED