EUVIMEDEUVIMED entdecken
Crossref

Hierarchical Token-Aware Cross-Modality Reconstruction for Visible-Infrared Person Re-Identification

Si Chen, Liuxiang Qiu, Da-Han Wang, Wentao Zhu, Yang Hua, Yan Yan

IEEE Transactions on Multimedia · 2025 · 27 · 7012-7027

DOI: 10.1109/tmm.2025.3590933 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Multimedia, 2025. Verfasst von Si Chen und 5 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 74 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Xiamen, Unveiling and Leading Projects of Xiamen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visible-infrared person re-identification (VI-ReID) aims to query the same pedestrian’s visible (infrared) images in the gallery set from the infrared (visible) images. VI-ReID not only needs to deal with the challenging factors like pose variation and occlusion, but also requires handling the large modality discrepancy. Previous methods mainly focus on learning single-scale modality-shared features and do not effectively explore the multi-scale features of two modalities from both short-range and long-range perspectives. In order to solve these problems, this paper proposes a novel Hierarchical Token-Aware Cross-Modality Reconstruction (HTCR) network to significantly mitigate the modality discrepancy for effective VI-ReID. The HTCR network consists of two main components, i.e., Hierarchical Token-aware Fusion (HTF) and Cross-modality Feature Reconstruction (CFR). The HTF module first bidirectionally exchanges the short-range and long-range multi-scale modality-shared features with a few learnable tokens to achieve discriminative pedestrian features by making full use of the advantages of both Convolutional Neural Network (CNN) and Transformer. Moreover, the CFR module reconstructs global and local pedestrian features of one modality by using the token sequence of the other modality with multi-scale cues to further explore the relationship between the two distinct modalities and alleviate the modality discrepancy. In addition, the Modality-shared feature Reconstruction (MR) loss is leveraged to reduce the noises between the reconstructed and the target features. Experimental results indicate that the proposed HTCR can significantly improve the VI-ReID performance and outperform the state-of-the-art methods on the cross-modality SYSU-MM01, RegDB, and LLCM datasets.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Xiamen
  • Unveiling and Leading Projects of Xiamen
  • Major Science and Technology Plan Project on the Future Industry Fields of Xiamen City

Zitieren

APA

Chen, S., Qiu, L., Wang, D., Zhu, W., Hua, Y., & Yan, Y. (2025). Hierarchical Token-Aware Cross-Modality Reconstruction for Visible-Infrared Person Re-Identification. IEEE Transactions on Multimedia, 27, 7012-7027. https://doi.org/10.1109/tmm.2025.3590933

Vancouver

Chen S, Qiu L, Wang D, Zhu W, Hua Y, Yan Y. Hierarchical Token-Aware Cross-Modality Reconstruction for Visible-Infrared Person Re-Identification. IEEE Transactions on Multimedia. 2025;27:7012-7027. doi:10.1109/tmm.2025.3590933

BibTeX
@article{Chen2025,
  author = {Chen, Si and Qiu, Liuxiang and Wang, Da-Han and Zhu, Wentao and Hua, Yang and Yan, Yan},
  title = {Hierarchical Token-Aware Cross-Modality Reconstruction for Visible-Infrared Person Re-Identification},
  journal = {IEEE Transactions on Multimedia},
  year = {2025},
  volume = {27},
  pages = {7012--7027},
  doi = {10.1109/tmm.2025.3590933}
}
Kostenlos starten →