EUVIMEDEUVIMED entdecken
Crossref

Distribution-to-Points Matching for Image Text Retrieval

Zheng Wang, Xing Xu, Lei Zhu, Jingkuan Song, Yang Yang, Heng Tao Shen

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-06 · 48 (6) · 7097-7111

DOI: 10.1109/tpami.2026.3664613 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-06. Verfasst von Zheng Wang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 91 Literaturangaben. Gefördert von: National Natural Science Foundation of China, New Cornerstone Science Foundation, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Eliminating semantic discrepancy between different modalities is the ultimate goal of image text retrieval. However, most of the existing methods only focus on retrieval of the ground-truth instance while ignoring those semantically similar instances yet unlabeled as positives, which causes the phenomenon of one-to-many correspondence. The mainstream solutions of this research are mainly based on uncertainty learning and the exploration of one-to-many correspondence is still insufficient albeit their significant progress. Therefore, this work develops a novel Distribution-to-Points (termed D2P) matching mechanism for image-text retrieval to capture the one-to-many correspondence between multiple samples and a given query via hypergraph modeling. Specifically, a given query is first mapped as a probabilistic embedding to learn its true semantic distribution based on Mahalanobis distance. Then each candidate instance in a mini-batch is regarded as a hypergraph node with its mean semantics while a Gaussian query is modeled as a hyperedge to capture the semantic correlations beyond the pair between candidate points and the query. Moreover, an energy-based semantic modeling framework is developed to pull all similar candidates (not only the ground truth) close to their query while pushing those dissimilar ones far away. In the end, distribution-to-points matching is learned based on the similarity measurement over the Mahalanobis distance, which considers semantic variance to perform many-to-one correspondence well. Experimental results on several widely used datasets and under various evaluation metrics confirm our superiority and effectiveness in improving the retrieval ability of the baseline including ground-truth matching and semantic multiplicity for image text retrieval.

Förderung

  • National Natural Science Foundation of China
  • New Cornerstone Science Foundation
  • Fundamental Research Funds for the Central Universities
  • Guangdong Basic and Applied Basic Research Foundation

Zitieren

APA

Wang, Z., Xu, X., Zhu, L., Song, J., Yang, Y., & Shen, H. T. (2026). Distribution-to-Points Matching for Image Text Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(6), 7097-7111. https://doi.org/10.1109/tpami.2026.3664613

Vancouver

Wang Z, Xu X, Zhu L, Song J, Yang Y, Shen HT. Distribution-to-Points Matching for Image Text Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(6):7097-7111. doi:10.1109/tpami.2026.3664613

BibTeX
@article{Wang2026,
  author = {Wang, Zheng and Xu, Xing and Zhu, Lei and Song, Jingkuan and Yang, Yang and Shen, Heng Tao},
  title = {Distribution-to-Points Matching for Image Text Retrieval},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {6},
  pages = {7097--7111},
  doi = {10.1109/tpami.2026.3664613}
}
Kostenlos starten →