Cross-modality feature fusion for night pedestrian detection
Frontiers in Physics · 26.03.2024 · 12
DOI: 10.3389/fphy.2024.1356248 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Physics, online erschienen am 26.03.2024. Verfasst von Yong Feng und 3 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 79 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Night pedestrian detection with visible image only suffers from the dilemma of high miss rate due to poor illumination conditions. Cross-modality fusion can ameliorate this dilemma by providing complementary information to each other through infrared and visible images. In this paper, we propose a cross-modal fusion framework based on YOLOv5, which is aimed at addressing the challenges of night pedestrian detection under low-light conditions. The framework employs a dual-stream architecture that processes visible images and infrared images separately. Through the Cross-Modal Feature Rectification Module (CMFRM), visible and infrared features are finely tuned on a granular level, leveraging their spatial correlations to focus on complementary information and substantially reduce uncertainty and noise from different modalities. Additionally, we have introduced a two-stage Feature Fusion Module (FFM), with the first stage introducing a cross-attention mechanism for cross-modal global reasoning, and the second stage using a mixed channel embedding to produce enhanced feature outputs. Moreover, our method involves multi-dimensional interaction, not only correcting feature maps in terms of channel and spatial dimensions but also applying cross-attention at the sequence processing level, which is critical for the effective generalization of cross-modal feature combinations. In summary, our research significantly enhances the accuracy and robustness of nighttime pedestrian detection, offering new perspectives and technical pathways for visual information processing in low-light environments.
Zitieren
APA
Feng, Y., Luo, E., Lu, H., & Zhai, S. (2024). Cross-modality feature fusion for night pedestrian detection. Frontiers in Physics, 12. https://doi.org/10.3389/fphy.2024.1356248
Vancouver
Feng Y, Luo E, Lu H, Zhai S. Cross-modality feature fusion for night pedestrian detection. Frontiers in Physics. 2024;12. doi:10.3389/fphy.2024.1356248
BibTeX
@article{Feng2024,
author = {Feng, Yong and Luo, Enbo and Lu, Hai and Zhai, SuWei},
title = {Cross-modality feature fusion for night pedestrian detection},
journal = {Frontiers in Physics},
year = {2024},
volume = {12},
doi = {10.3389/fphy.2024.1356248}
}
EUVIMED