EUVIMEDEUVIMED entdecken
Crossref

MSCoTDet: Language-Driven Multi-Modal Fusion for Improved Multispectral Pedestrian Detection

Taeheon Kim, Sangyun Chung, Damin Yeom, Youngjoon Yu, Hak Gu Kim, Yong Man Ro

IEEE Transactions on Circuits and Systems for Video Technology · 2025-05 · 35 (5) · 5006-5021

DOI: 10.1109/tcsvt.2024.3524645 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-05. Verfasst von Taeheon Kim und 5 weiteren Autor:innen. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 97 Literaturangaben. Gefördert von: Center for Applied Research in Artificial Intelligence, Defense Acquisition Program Administration (DAPA) and Agency for Defense Development.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multispectral pedestrian detection is attractive for around-the-clock applications due to the complementary information between RGB and thermal modalities. However, current models often fail to detect pedestrians in certain cases (e.g., thermal-obscured pedestrians), particularly due to the modality bias learned from statistically biased datasets. In this paper, we investigate how to mitigate modality bias in multispectral pedestrian detection using a Large Language Model (LLM). Accordingly, we design a Multispectral Chain-of-Thought (MSCoT) prompting strategy, which prompts the LLM to perform multispectral pedestrian detection. Moreover, we propose a novel Multispectral Chain-of-Thought Detection (MSCoTDet) framework that integrates MSCoT prompting into multispectral pedestrian detection. To this end, we design a Language-driven Multi-modal Fusion (LMF) strategy that enables fusing the outputs of MSCoT prompting with the detection results of vision-based multispectral pedestrian detection models. Extensive experiments validate that MSCoTDet effectively mitigates modality biases and improves multispectral pedestrian detection.

Förderung

  • Center for Applied Research in Artificial Intelligence
  • Defense Acquisition Program Administration (DAPA) and Agency for Defense Development

Zitieren

APA

Kim, T., Chung, S., Yeom, D., Yu, Y., Kim, H. G., & Ro, Y. M. (2025). MSCoTDet: Language-Driven Multi-Modal Fusion for Improved Multispectral Pedestrian Detection. IEEE Transactions on Circuits and Systems for Video Technology, 35(5), 5006-5021. https://doi.org/10.1109/tcsvt.2024.3524645

Vancouver

Kim T, Chung S, Yeom D, Yu Y, Kim HG, Ro YM. MSCoTDet: Language-Driven Multi-Modal Fusion for Improved Multispectral Pedestrian Detection. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(5):5006-5021. doi:10.1109/tcsvt.2024.3524645

BibTeX
@article{Kim2025,
  author = {Kim, Taeheon and Chung, Sangyun and Yeom, Damin and Yu, Youngjoon and Kim, Hak Gu and Ro, Yong Man},
  title = {MSCoTDet: Language-Driven Multi-Modal Fusion for Improved Multispectral Pedestrian Detection},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {5},
  pages = {5006--5021},
  doi = {10.1109/tcsvt.2024.3524645}
}
Kostenlos starten →