EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

YT-SWIN: Context-Adaptive Spectral Fusion for Cross-Modal UAV Remote Sensing

Qurat Ul Ain, Kewen Xia, Ali Zaib Khan

IEEE Open Journal of Vehicular Technology · 2026 · 7 · 1832-1844

DOI: 10.1109/ojvt.2026.3702128 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Open Journal of Vehicular Technology, 2026. Verfasst von Qurat Ul Ain und 2 weiteren Autor:innen. Die Arbeit nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Hebei Province Natural Science Foundation, Key Research and Development Project from Hebei Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurate object detection in multimodal remote sensing images remains a significant challenge for autonomous all-weather systems, necessitating adaptive integration of complementary RGB and infrared (IR) data under diverse environmental conditions. Existing fusion approaches often rely on fixed, manually designed strategies or complex multi-expert frameworks, which lack the flexibility to dynamically achieve fusion policies in response to real-time sensor degradation. To address these limitations, we propose YT-SWIN, a novel adaptive fusion framework that synergistically combines the YOLOv10n backbone with a Swin Transformer-based attention mechanism to achieve environment-aware multimodal perception. Our work introduces three core modules: first, Self-Supervised Environment Inference Module that learns to characterize weather and illumination conditions directly from input data, eliminating the reliance on labeled environmental metadata. Second, a Learnable Frequency Decomposition Block that constructs optimized spectral representations through data-adaptive filtering, enhancing multimodal feature representation. Third, a Hierarchical Cross-Modal Attention Network (HCAN) built upon lightweight Swin Transformer blocks dynamically gates and fuses RGB and IR features across multiple spatial and frequency scales, conditioned on the inferred environmental context. Additionally, an adaptive Channel Pruning Strategy integrated into the multi-scale feature pyramid maintains real-time inference speeds without compromising detection accuracy. Extensive experiments demonstrate that YT-SWIN achieves state-of-the-art performance, attaining mAP scores of 83.08% on the DroneVehicle dataset and 77.4% on the VEDAI dataset while operating in real time. The framework demonstrates strong generalization capability across environments, establishing a new paradigm for efficient, adaptive, and robust multimodal perception in unstructured outdoor scenarios.

Förderung

  • National Natural Science Foundation of China
  • Hebei Province Natural Science Foundation
  • Key Research and Development Project from Hebei Province

Zitieren

APA

Ain, Q. U., Xia, K., & Khan, A. Z. (2026). YT-SWIN: Context-Adaptive Spectral Fusion for Cross-Modal UAV Remote Sensing. IEEE Open Journal of Vehicular Technology, 7, 1832-1844. https://doi.org/10.1109/ojvt.2026.3702128

Vancouver

Ain QU, Xia K, Khan AZ. YT-SWIN: Context-Adaptive Spectral Fusion for Cross-Modal UAV Remote Sensing. IEEE Open Journal of Vehicular Technology. 2026;7:1832-1844. doi:10.1109/ojvt.2026.3702128

BibTeX
@article{Ain2026,
  author = {Ain, Qurat Ul and Xia, Kewen and Khan, Ali Zaib},
  title = {YT-SWIN: Context-Adaptive Spectral Fusion for Cross-Modal UAV Remote Sensing},
  journal = {IEEE Open Journal of Vehicular Technology},
  year = {2026},
  volume = {7},
  pages = {1832--1844},
  doi = {10.1109/ojvt.2026.3702128}
}
Kostenlos starten →