INSANet: INtra-INter Spectral Attention Network for Effective Feature Fusion of Multispectral Pedestrian Detection
Sensors · 10.02.2024 · 24 (4) · 1168
DOI: 10.3390/s24041168 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 10.02.2024. Verfasst von Sangin Lee und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Software, Sejong University, Seoul 05006, Republic of Korea. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science and ICT, Institute of Information & communications Technology Planning & Evaluation.
Abstract (Originalsprache)
Pedestrian detection is a critical task for safety-critical systems, but detecting pedestrians is challenging in low-light and adverse weather conditions. Thermal images can be used to improve robustness by providing complementary information to RGB images. Previous studies have shown that multi-modal feature fusion using convolution operation can be effective, but such methods rely solely on local feature correlations, which can degrade the performance capabilities. To address this issue, we propose an attention-based novel fusion network, referred to as INSANet (INtra-INter Spectral Attention Network), that captures global intra- and inter-information. It consists of intra- and inter-spectral attention blocks that allow the model to learn mutual spectral relationships. Additionally, we identified an imbalance in the multispectral dataset caused by several factors and designed an augmentation strategy that mitigates concentrated distributions and enables the model to learn the diverse locations of pedestrians. Extensive experiments demonstrate the effectiveness of the proposed methods, which achieve state-of-the-art performance on the KAIST dataset and LLVIP dataset. Finally, we conduct a regional performance evaluation to demonstrate the effectiveness of our proposed network in various regions.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Software, Sejong University, Seoul 05006, Republic of Korea.
- Department of Convergence Engineering for Intelligent Drone, Sejong University, Seoul 05006, Republic of Korea.
- NAVER LABS, Seongnam 13561, Republic of Korea.
ORCID-Profile: Sangin Lee, Taejoo Kim, Jeongmin Shin, Namil Kim, Yukyung Choi
Förderung
- Ministry of Science and ICT
- Institute of Information & communications Technology Planning & Evaluation
Zitieren
APA
Lee, S., Kim, T., Shin, J., Kim, N., & Choi, Y. (2024). INSANet: INtra-INter Spectral Attention Network for Effective Feature Fusion of Multispectral Pedestrian Detection. Sensors, 24(4), 1168. https://doi.org/10.3390/s24041168
Vancouver
Lee S, Kim T, Shin J, Kim N, Choi Y. INSANet: INtra-INter Spectral Attention Network for Effective Feature Fusion of Multispectral Pedestrian Detection. Sensors. 2024;24(4):1168. doi:10.3390/s24041168
BibTeX
@article{Lee2024,
author = {Lee, Sangin and Kim, Taejoo and Shin, Jeongmin and Kim, Namil and Choi, Yukyung},
title = {INSANet: INtra-INter Spectral Attention Network for Effective Feature Fusion of Multispectral Pedestrian Detection},
journal = {Sensors},
year = {2024},
volume = {24},
number = {4},
pages = {1168},
doi = {10.3390/s24041168}
}
EUVIMED