EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Cross-modality feature fusion for night pedestrian detection

Yong Feng, Enbo Luo, Hai Lu, SuWei Zhai

Frontiers in Physics · 26.03.2024 · 12

DOI: 10.3389/fphy.2024.1356248 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Physics, online erschienen am 26.03.2024. Verfasst von Yong Feng und 3 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 79 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Night pedestrian detection with visible image only suffers from the dilemma of high miss rate due to poor illumination conditions. Cross-modality fusion can ameliorate this dilemma by providing complementary information to each other through infrared and visible images. In this paper, we propose a cross-modal fusion framework based on YOLOv5, which is aimed at addressing the challenges of night pedestrian detection under low-light conditions. The framework employs a dual-stream architecture that processes visible images and infrared images separately. Through the Cross-Modal Feature Rectification Module (CMFRM), visible and infrared features are finely tuned on a granular level, leveraging their spatial correlations to focus on complementary information and substantially reduce uncertainty and noise from different modalities. Additionally, we have introduced a two-stage Feature Fusion Module (FFM), with the first stage introducing a cross-attention mechanism for cross-modal global reasoning, and the second stage using a mixed channel embedding to produce enhanced feature outputs. Moreover, our method involves multi-dimensional interaction, not only correcting feature maps in terms of channel and spatial dimensions but also applying cross-attention at the sequence processing level, which is critical for the effective generalization of cross-modal feature combinations. In summary, our research significantly enhances the accuracy and robustness of nighttime pedestrian detection, offering new perspectives and technical pathways for visual information processing in low-light environments.

Zitieren

APA

Feng, Y., Luo, E., Lu, H., & Zhai, S. (2024). Cross-modality feature fusion for night pedestrian detection. Frontiers in Physics, 12. https://doi.org/10.3389/fphy.2024.1356248

Vancouver

Feng Y, Luo E, Lu H, Zhai S. Cross-modality feature fusion for night pedestrian detection. Frontiers in Physics. 2024;12. doi:10.3389/fphy.2024.1356248

BibTeX
@article{Feng2024,
  author = {Feng, Yong and Luo, Enbo and Lu, Hai and Zhai, SuWei},
  title = {Cross-modality feature fusion for night pedestrian detection},
  journal = {Frontiers in Physics},
  year = {2024},
  volume = {12},
  doi = {10.3389/fphy.2024.1356248}
}
Kostenlos starten →