EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification

Wenjie Pan iD, Linhan Huang, Jianbao Liang, Lan Hong, Jianqing Zhu iD

Sensors · 23.04.2023 · 23 (9) · 4206

DOI: 10.3390/s23094206 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 23.04.2023. Verfasst von Wenjie Pan und 4 weiteren Autor:innen. Beteiligte Einrichtung: College of Engineering, Huaqiao University, Quanzhou 362021, China. Die Arbeit wurde bisher 33-mal zitiert (Crossref) und nennt 66 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Natural Science Foundation for Outstanding Young Scholars of Fujian Province, Collaborative Innovation Platform Project of Fuzhou-Xiamen-Quanzhou National Independent Innovation Demonstration Zone.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-modal (i.e., visible, near-infrared, and thermal-infrared) vehicle re-identification has good potential to search vehicles of interest in low illumination. However, due to the fact that different modalities have varying imaging characteristics, a proper multi-modal complementary information fusion is crucial to multi-modal vehicle re-identification. For that, this paper proposes a progressively hybrid transformer (PHT). The PHT method consists of two aspects: random hybrid augmentation (RHA) and a feature hybrid mechanism (FHM). Regarding RHA, an image random cropper and a local region hybrider are designed. The image random cropper simultaneously crops multi-modal images of random positions, random numbers, random sizes, and random aspect ratios to generate local regions. The local region hybrider fuses the cropped regions to let regions of each modal bring local structural characteristics of all modalities, mitigating modal differences at the beginning of feature learning. Regarding the FHM, a modal-specific controller and a modal information embedding are designed to effectively fuse multi-modal information at the feature level. Experimental results show the proposed method wins the state-of-the-art method by a larger 2.7% mAP on RGBNT100 and a larger 6.6% mAP on RGBN300, demonstrating that the proposed method can learn multi-modal complementary information effectively.

Schlagwörter

Stichwörter der Autor:innen

multi-modal imagetransformervehicle re-identification

Autor:innen und Institutionen

  • College of Engineering, Huaqiao University, Quanzhou 362021, China.
  • Xiamen Yealink Network Technology Company Limited, No. 666, Hu'an Road, High-Tech Park, Huli District, Xiamen 361015, China.

ORCID-Profile: Wenjie Pan, Jianqing Zhu

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation for Outstanding Young Scholars of Fujian Province
  • Collaborative Innovation Platform Project of Fuzhou-Xiamen-Quanzhou National Independent Innovation Demonstration Zone

Zitieren

APA

Pan, W., Huang, L., Liang, J., Hong, L., & Zhu, J. (2023). Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification. Sensors, 23(9), 4206. https://doi.org/10.3390/s23094206

Vancouver

Pan W, Huang L, Liang J, Hong L, Zhu J. Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification. Sensors. 2023;23(9):4206. doi:10.3390/s23094206

BibTeX
@article{Pan2023,
  author = {Pan, Wenjie and Huang, Linhan and Liang, Jianbao and Hong, Lan and Zhu, Jianqing},
  title = {Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {9},
  pages = {4206},
  doi = {10.3390/s23094206}
}
Kostenlos starten →