Weakly Supervised Hashing with Reconstructive Cross-modal Attention
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.11.2023 · 19 (6) · 1-19
DOI: 10.1145/3589185 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 12.07.2023. Verfasst von Yongchao Du und 4 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 50 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, GPU cluster built by MCC Lab of Information Science and Technology Institution.
Abstract (Originalsprache)
On many popular social websites, images are usually associated with some meta-data such as textual tags, which involve semantic information relevant to the image and can be used to supervise the representation learning for image retrieval. However, these user-provided tags are usually polluted by noise, therefore the main challenge lies in mining the potential useful information from those noisy tags. Many previous works simply treat different tags equally to generate supervision, which will inevitably distract the network learning. To this end, we propose a new framework, termed as Weakly Supervised Hashing with Reconstructive Cross-modal Attention (WSHRCA), to learn compact visual-semantic representation with more reliable supervision for retrieval task. Specifically, for each image-tag pair, the weak supervision from tags is refined by cross-modal attention, which takes image feature as query to aggregate the most content-relevant tags. Therefore, tags with relevant content will be more prominent while noisy tags will be suppressed, which provides more accurate supervisory information. To improve the effectiveness of hash learning, the image embedding in WSHRCA is reconstructed from hash code, which is further optimized by cross-modal constraint and explicitly improves hash learning. The experiments on two widely-used datasets demonstrate the effectiveness of our proposed method for weakly-supervised image retrieval. The code is available at https://github.com/duyc168/weakly-supervised-hashing .
Förderung
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities
- GPU cluster built by MCC Lab of Information Science and Technology Institution
- Supercomputing Center of the USTC
Zitieren
APA
Du, Y., Wang, M., Lu, Z., Zhou, W., & Li, H. (2023). Weakly Supervised Hashing with Reconstructive Cross-modal Attention. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(6), 1-19. https://doi.org/10.1145/3589185
Vancouver
Du Y, Wang M, Lu Z, Zhou W, Li H. Weakly Supervised Hashing with Reconstructive Cross-modal Attention. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(6):1-19. doi:10.1145/3589185
BibTeX
@article{Du2023,
author = {Du, Yongchao and Wang, Min and Lu, Zhenbo and Zhou, Wengang and Li, Houqiang},
title = {Weakly Supervised Hashing with Reconstructive Cross-modal Attention},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2023},
volume = {19},
number = {6},
pages = {1--19},
doi = {10.1145/3589185}
}
EUVIMED