Graph-Based Progressive Fusion Network for Multi-Modality Vehicle Re-Identification
IEEE Transactions on Intelligent Transportation Systems · 2023-11 · 24 (11) · 12431-12447
DOI: 10.1109/tits.2023.3285758 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Intelligent Transportation Systems, 2023-11. Verfasst von Qiaolin He und 3 weiteren Autor:innen. Die Arbeit wurde bisher 40-mal zitiert (Crossref) und nennt 79 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Vehicle re-identification (Re-ID) is a critical task in intelligent transportation, aiming to match vehicle images of the same identity captured by non-overlapping cameras. However, it is difficult to achieve satisfactory results based on RGB images alone in darkness. Therefore, it is of great importance to consider multi-modality vehicle re-identification. Currently, the proposed works deal with different modality features through direct summation and fusion based on heat map, which however ignores the relationship between them. Meanwhile, there is a huge gap between the different modalities, which needs to be reduced. In this paper, to solve the above two problems, we propose a Graph-based Progressive Fusion Network (GPFNet) using a graph convolutional network to adaptively fuse multi-modality features in an end-to-end learning framework. GPFNet consists of a CNN feature extraction module (FEM), a GCN feature fusion module (FFM), and a loss function module (LFM). Firstly, in FEM, we employ a multi-stream network architecture to extract single-modality features and common-modality features and employ a random modality substitution module to extract mixed-modality features. Secondly, in FFM, we design an efficient graph structure to associate the features of different modalities and adopt a progressive two-stage strategy to fuse them. Finally, in LFM, we use GCN-aware multi-modality loss to constrain the features. For reducing modality differences and contributing better initial mixed-modality features to FFM, we propose random modality substitution as a data enhancement method for multi-modality datasets. Extensive experiments on multi-modality vehicle Re-ID datasets RGBN300 and RGBNT100 show that our model achieves state-of-the-art performance.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
He, Q., Lu, Z., Wang, Z., & Hu, H. (2023). Graph-Based Progressive Fusion Network for Multi-Modality Vehicle Re-Identification. IEEE Transactions on Intelligent Transportation Systems, 24(11), 12431-12447. https://doi.org/10.1109/tits.2023.3285758
Vancouver
He Q, Lu Z, Wang Z, Hu H. Graph-Based Progressive Fusion Network for Multi-Modality Vehicle Re-Identification. IEEE Transactions on Intelligent Transportation Systems. 2023;24(11):12431-12447. doi:10.1109/tits.2023.3285758
BibTeX
@article{He2023,
author = {He, Qiaolin and Lu, Zefeng and Wang, Zihan and Hu, Haifeng},
title = {Graph-Based Progressive Fusion Network for Multi-Modality Vehicle Re-Identification},
journal = {IEEE Transactions on Intelligent Transportation Systems},
year = {2023},
volume = {24},
number = {11},
pages = {12431--12447},
doi = {10.1109/tits.2023.3285758}
}
EUVIMED