EUVIMEDEUVIMED entdecken
Crossref

Trustworthy Visual-Textual Retrieval

Yang Qin, Lifu Huang, Dezhong Peng, Bohan Jiang, Joey Tianyi Zhou, Xi Peng, Peng Hu

IEEE Transactions on Image Processing · 2025 · 34 · 4515-4526

DOI: 10.1109/tip.2025.3587575 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Yang Qin und 6 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 73 Literaturangaben. Gefördert von: National Key Research and Development Program of China, NSFC, Sichuan Science and Technology Planning Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual-textual retrieval, as a link between computer vision and natural language processing, aims at jointly learning visual-semantic relevance to bridge the heterogeneity gap across visual and textual spaces. Existing methods conduct retrieval only relying on the ranking of pairwise similarities, but they cannot self-evaluate the uncertainty of retrieved results, resulting in unreliable retrieval and hindering interpretability. To address this problem, we propose a novel Trust-Consistent Learning framework (TCL) to endow visual-textual retrieval with uncertainty evaluation for trustworthy retrieval. More specifically, TCL first models the matching evidence according to cross-modal similarity to estimate the uncertainty for cross-modal uncertainty-aware learning. Second, a simple yet effective consistency module is presented to enforce the subjective opinions of bidirectional learning to be consistent for high reliability and accuracy. Finally, extensive experiments are conducted to demonstrate the superiority and generalizability of TCL on six widely-used benchmark datasets, i.e., Flickr30K, MS-COCO, MSVD, MSR-VTT, ActivityNet, and DiDeMo. Furthermore, some qualitative experiments are carried out to provide comprehensive and insightful analyses for trustworthy visual-textual retrieval, verifying the reliability and interoperability of TCL. The code is available in https://github.com/QinYang79/TCL.

Förderung

  • National Key Research and Development Program of China
  • NSFC
  • Sichuan Science and Technology Planning Project
  • System of Systems and Artificial Intelligence Laboratory Pioneer Fund
  • Fundamental Research Funds for the Central Universities
  • Chengdu Municipal Science and Technology Program

Zitieren

APA

Qin, Y., Huang, L., Peng, D., Jiang, B., Tianyi Zhou, J., Peng, X., & Hu, P. (2025). Trustworthy Visual-Textual Retrieval. IEEE Transactions on Image Processing, 34, 4515-4526. https://doi.org/10.1109/tip.2025.3587575

Vancouver

Qin Y, Huang L, Peng D, Jiang B, Tianyi Zhou J, Peng X, et al. Trustworthy Visual-Textual Retrieval. IEEE Transactions on Image Processing. 2025;34:4515-4526. doi:10.1109/tip.2025.3587575

BibTeX
@article{Qin2025,
  author = {Qin, Yang and Huang, Lifu and Peng, Dezhong and Jiang, Bohan and Tianyi Zhou, Joey and Peng, Xi and Hu, Peng},
  title = {Trustworthy Visual-Textual Retrieval},
  journal = {IEEE Transactions on Image Processing},
  year = {2025},
  volume = {34},
  pages = {4515--4526},
  doi = {10.1109/tip.2025.3587575}
}
Kostenlos starten →