Trustworthy Visual-Textual Retrieval
IEEE Transactions on Image Processing · 2025 · 34 · 4515-4526
DOI: 10.1109/tip.2025.3587575 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Yang Qin und 6 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 73 Literaturangaben. Gefördert von: National Key Research and Development Program of China, NSFC, Sichuan Science and Technology Planning Project.
Abstract (Originalsprache)
Visual-textual retrieval, as a link between computer vision and natural language processing, aims at jointly learning visual-semantic relevance to bridge the heterogeneity gap across visual and textual spaces. Existing methods conduct retrieval only relying on the ranking of pairwise similarities, but they cannot self-evaluate the uncertainty of retrieved results, resulting in unreliable retrieval and hindering interpretability. To address this problem, we propose a novel Trust-Consistent Learning framework (TCL) to endow visual-textual retrieval with uncertainty evaluation for trustworthy retrieval. More specifically, TCL first models the matching evidence according to cross-modal similarity to estimate the uncertainty for cross-modal uncertainty-aware learning. Second, a simple yet effective consistency module is presented to enforce the subjective opinions of bidirectional learning to be consistent for high reliability and accuracy. Finally, extensive experiments are conducted to demonstrate the superiority and generalizability of TCL on six widely-used benchmark datasets, i.e., Flickr30K, MS-COCO, MSVD, MSR-VTT, ActivityNet, and DiDeMo. Furthermore, some qualitative experiments are carried out to provide comprehensive and insightful analyses for trustworthy visual-textual retrieval, verifying the reliability and interoperability of TCL. The code is available in https://github.com/QinYang79/TCL.
Förderung
- National Key Research and Development Program of China
- NSFC
- Sichuan Science and Technology Planning Project
- System of Systems and Artificial Intelligence Laboratory Pioneer Fund
- Fundamental Research Funds for the Central Universities
- Chengdu Municipal Science and Technology Program
Zitieren
APA
Qin, Y., Huang, L., Peng, D., Jiang, B., Tianyi Zhou, J., Peng, X., & Hu, P. (2025). Trustworthy Visual-Textual Retrieval. IEEE Transactions on Image Processing, 34, 4515-4526. https://doi.org/10.1109/tip.2025.3587575
Vancouver
Qin Y, Huang L, Peng D, Jiang B, Tianyi Zhou J, Peng X, et al. Trustworthy Visual-Textual Retrieval. IEEE Transactions on Image Processing. 2025;34:4515-4526. doi:10.1109/tip.2025.3587575
BibTeX
@article{Qin2025,
author = {Qin, Yang and Huang, Lifu and Peng, Dezhong and Jiang, Bohan and Tianyi Zhou, Joey and Peng, Xi and Hu, Peng},
title = {Trustworthy Visual-Textual Retrieval},
journal = {IEEE Transactions on Image Processing},
year = {2025},
volume = {34},
pages = {4515--4526},
doi = {10.1109/tip.2025.3587575}
}
EUVIMED