EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Uni–MPedFormer: towards a unified transformer–based framework for multispectral pedestrian detection

Yan Li, Chaoqi Yan, Cunfei Zhao, Jianbo Song

Journal of King Saud University Computer and Information Sciences · 2026-08 · 38 (6)

DOI: 10.1007/s44443-026-00951-9 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of King Saud University Computer and Information Sciences, online erschienen am 26.06.2026. Verfasst von Yan Li und 3 weiteren Autor:innen. Die Arbeit nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Natural Science Foundation of Shandong Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Robust pedestrian detection in real-world scenes remains difficult because of scale variation, body occlusion, and degraded nighttime visibility. RGB-T image pairs offer complementary spectral cues that can mitigate these problems, but most existing methods address each challenge in isolation rather than within a single architecture. We introduce Uni-MPedFormer, a unified transformer-based multispectral pedestrian detection framework that handles all three modality-shared difficulties within an end-to-end trainable network. The framework consists of three components. First, a Cross-modality Information Fusion Transformer (CIFT) encoder uses the global receptive field of self-attention to capture intra- and inter-modal dependencies between RGB and thermal streams. A Feature Correlation Recalibration (FCR) inner-attention block then enforces neighbourhood-consistent query–key correlations and suppresses spurious responses. Second, a U-shaped Feature Aggregation (UFA) decoder merges multi-scale hierarchical representations through lateral, top-down, and bottom-up pathways, sharpening spatial localisation across pedestrian scales. Third, a lightweight Illumination-aware Weight Generation (IWG) module estimates scene illumination and dynamically re-weights each modality’s contribution to the final detection score. Experiments on five public benchmarks—KAIST, CVC-14, FLIR-ADAS, LLVIP, and UTokyo—and comparisons with recent multimodal detectors show that Uni-MPedFormer achieves competitive or superior accuracy while maintaining a favourable accuracy–speed trade-off (25 FPS on a single 2080Ti). We further report an explicit complexity analysis (parameters, FLOPs, peak memory), a scalability study at higher input resolutions, a sensitivity study on the FCR reduced dimension D, a robustness analysis under simulated RGB-T misalignment, and an explicit module-to-challenge correspondence between CIFT and occlusion, UFA and small-scale pedestrians, and IWG and nighttime detection.

Förderung

  • Natural Science Foundation of Shandong Province

Zitieren

APA

Li, Y., Yan, C., Zhao, C., & Song, J. (2026). Uni–MPedFormer: towards a unified transformer–based framework for multispectral pedestrian detection. Journal of King Saud University Computer and Information Sciences, 38(6). https://doi.org/10.1007/s44443-026-00951-9

Vancouver

Li Y, Yan C, Zhao C, Song J. Uni–MPedFormer: towards a unified transformer–based framework for multispectral pedestrian detection. Journal of King Saud University Computer and Information Sciences. 2026;38(6). doi:10.1007/s44443-026-00951-9

BibTeX
@article{Li2026,
  author = {Li, Yan and Yan, Chaoqi and Zhao, Cunfei and Song, Jianbo},
  title = {Uni–MPedFormer: towards a unified transformer–based framework for multispectral pedestrian detection},
  journal = {Journal of King Saud University Computer and Information Sciences},
  year = {2026},
  volume = {38},
  number = {6},
  doi = {10.1007/s44443-026-00951-9}
}
Kostenlos starten →