EUVIMEDEUVIMED entdecken
Crossref

Text-and-Image Learning Transformer for Cross-Modal Person Re-Identification

Tinghui Wu, Shuhe Zhang, Dihu Chen, Haifeng Hu

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.01.2025 · 21 (1) · 1-18

DOI: 10.1145/3686160 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 20.12.2024. Verfasst von Tinghui Wu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 74 Literaturangaben. Gefördert von: Science and Technology Program of Guangdong Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Text-based person re-identification aims to find the target person from a large pedestrian gallery with the given natural language description. Previous works mainly focus on embedding salient textual and visual representations in a common latent space by utilizing the dual-path structure or parameter-shared network. However, they still lack the ability to effectively extract fine-grained unimodal features as well as fuse the cross-modal data, leading to the increase of misaligned cases. To settle these issues, we propose a text-and-image implicit learning Transformer (TILT) to eliminate textual anisotropy and enhance the cross-modal alignment from both domains based on the bi-direction multi-modal encoders. Specifically, we apply the pre-trained multi-modal embedding module to overcome the unimodal anisotropy problem with contrastive learning, and map fine-grained features with dual encoder in bi-directional masking. Then, we design the cross-modal interaction encoder to comprehensively mine implicit cross-modal relations by reconstructing masked tokens, and fuse rich multi-modal knowledge in a common space. In addition, the cross-modal similarity matching module is proposed to optimize the intra-domain classification and decrease the inter-domain divergence. Extensive experiments are conducted on three public benchmarks CUHK-PEDES, ICFG-PEDES, and RSTPReid to verify the effectiveness of our proposed framework. Results prove that our model outperforms state-of-the-art methods on all metrics.

Förderung

  • Science and Technology Program of Guangdong Province

Zitieren

APA

Wu, T., Zhang, S., Chen, D., & Hu, H. (2025). Text-and-Image Learning Transformer for Cross-Modal Person Re-Identification. ACM Transactions on Multimedia Computing, Communications, and Applications, 21(1), 1-18. https://doi.org/10.1145/3686160

Vancouver

Wu T, Zhang S, Chen D, Hu H. Text-and-Image Learning Transformer for Cross-Modal Person Re-Identification. ACM Transactions on Multimedia Computing, Communications, and Applications. 2025;21(1):1-18. doi:10.1145/3686160

BibTeX
@article{Wu2025,
  author = {Wu, Tinghui and Zhang, Shuhe and Chen, Dihu and Hu, Haifeng},
  title = {Text-and-Image Learning Transformer for Cross-Modal Person Re-Identification},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2025},
  volume = {21},
  number = {1},
  pages = {1--18},
  doi = {10.1145/3686160}
}
Kostenlos starten →