EUVIMEDPoznaj EUVIMED
Crossref

VGSG: Vision-Guided Semantic-Group Network for Text-Based Person Search

Shuting He, Hao Luo, Wei Jiang, Xudong Jiang, Henghui Ding

IEEE Transactions on Image Processing · 2024 · 33 · 163-176

DOI: 10.1109/tip.2023.3337653 ↗

At a glance

Journal article in IEEE Transactions on Image Processing, 2024. Written by Shuting He and 4 co-authors. Cited 116 times so far (Crossref); 71 references. Funded by: National Natural Science Foundation of China, Research Grant of Nanyang Technological University (NTU) Presidential Postdoctoral Fellowship.

AbstraktDane bibliograficzneCiteRelated studies

Abstract (original language)

Text-based Person Search (TBPS) aims to retrieve images of target pedestrian indicated by textual descriptions. It is essential for TBPS to extract fine-grained local features and align them crossing modality. Existing methods utilize external tools or heavy cross-modal interaction to achieve explicit alignment of cross-modal fine-grained features, which is inefficient and time-consuming. In this work, we propose a Vision-Guided Semantic-Group Network (VGSG) for text-based person search to extract well-aligned fine-grained visual and textual features. In the proposed VGSG, we develop a Semantic-Group Textual Learning (SGTL) module and a Vision-guided Knowledge Transfer (VGKT) module to extract textual local features under the guidance of visual local clues. In SGTL, in order to obtain the local textual representation, we group textual features from the channel dimension based on the semantic cues of language expression, which encourages similar semantic patterns to be grouped implicitly without external tools. In VGKT, a vision-guided attention is employed to extract visual-related textual features, which are inherently aligned with visual cues and termed vision-guided textual features. Furthermore, we design a relational knowledge transfer, including a vision-language similarity transfer and a class probability transfer, to adaptively propagate information of the vision-guided textual features to semantic-group textual features. With the help of relational knowledge transfer, VGKT is capable of aligning semantic-group textual features with corresponding visual features without external tools and complex pairwise interaction. Experimental results on two challenging benchmarks demonstrate its superiority over state-of-the-art methods.

Funding

  • National Natural Science Foundation of China
  • Research Grant of Nanyang Technological University (NTU) Presidential Postdoctoral Fellowship

Cite

APA

He, S., Luo, H., Jiang, W., Jiang, X., & Ding, H. (2024). VGSG: Vision-Guided Semantic-Group Network for Text-Based Person Search. IEEE Transactions on Image Processing, 33, 163-176. https://doi.org/10.1109/tip.2023.3337653

Vancouver

He S, Luo H, Jiang W, Jiang X, Ding H. VGSG: Vision-Guided Semantic-Group Network for Text-Based Person Search. IEEE Transactions on Image Processing. 2024;33:163-176. doi:10.1109/tip.2023.3337653

BibTeX
@article{He2024,
  author = {He, Shuting and Luo, Hao and Jiang, Wei and Jiang, Xudong and Ding, Henghui},
  title = {VGSG: Vision-Guided Semantic-Group Network for Text-Based Person Search},
  journal = {IEEE Transactions on Image Processing},
  year = {2024},
  volume = {33},
  pages = {163--176},
  doi = {10.1109/tip.2023.3337653}
}
Start for free →