Multimodal Uncertainty-Aware Gating Fusion and Iterative Feedback Refinement for HSI-LiDAR Open-Set Classification
Remote Sensing · 12.06.2026 · 18 (12) · 1963
DOI: 10.3390/rs18121963 ↗Kurz gefasst
Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 12.06.2026. Verfasst von Davaajargal Myagmarsuren und 2 weiteren Autor:innen. Die Arbeit nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the Key Research and Development Plan Project of Heilongjiang, the National Key Support Project for Foreign Experts of Northeast Special Project.
Abstract (Originalsprache)
Open-set classification for remote sensing requires models that simultaneously achieve high accuracy on known land-cover types and reliably detect novel classes absent from the training distribution—a capability essential for real-world deployment where new classes routinely emerge. Existing multimodal fusion approaches for hyperspectral imagery (HSI) and LiDAR are primarily designed for closed-set scenarios and lack robust uncertainty modeling for unknown detection. We propose a post hoc calibrated multimodal open-set framework with three tightly integrated components. First, an Uncertainty-Aware Gating Fusion (UAGF) module dynamically weights HSI and LiDAR features per sample based on modality reliability and produces a gating uncertainty signal reflecting fusion confidence. Second, an Iterative Feedback Refinement (IFR) module progressively refines fused representations over multiple iterations and captures convergence dynamics, where stable convergence indicates known samples while high feature-change variance identifies potential unknowns. Third, a compact two-signal open-set detector combines gating uncertainty and refinement variance through an EVT (Weibull)-based post hoc calibration mechanism fitted exclusively on known validation samples. The framework follows a strict zero-unknown-supervision protocol: the multimodal backbone is trained using only known-class samples, and the open-set decision threshold is derived solely from the known validation score distribution. This design decouples representation of learning from open-set decision learning, improving robustness and avoiding the objective conflicts that arise in joint training. Comprehensive experiments on three benchmark datasets—Houston2013, Muufl, and Augsburg—demonstrate that the proposed method achieves 92.79%, 84.47%, and 80.99% overall accuracy and 76.48%, 63.91%, and 56.81% unknown accuracy, outperforming the closest multimodal competitor HyLiOSR by up to 32.4 pp in unknown accuracy while maintaining competitive closed-set performance.
Förderung
- the Key Research and Development Plan Project of Heilongjiang
- the National Key Support Project for Foreign Experts of Northeast Special Project
Zitieren
APA
Myagmarsuren, D., Wu, H., & Wang, A. (2026). Multimodal Uncertainty-Aware Gating Fusion and Iterative Feedback Refinement for HSI-LiDAR Open-Set Classification. Remote Sensing, 18(12), 1963. https://doi.org/10.3390/rs18121963
Vancouver
Myagmarsuren D, Wu H, Wang A. Multimodal Uncertainty-Aware Gating Fusion and Iterative Feedback Refinement for HSI-LiDAR Open-Set Classification. Remote Sensing. 2026;18(12):1963. doi:10.3390/rs18121963
BibTeX
@article{Myagmarsuren2026,
author = {Myagmarsuren, Davaajargal and Wu, Haibin and Wang, Aili},
title = {Multimodal Uncertainty-Aware Gating Fusion and Iterative Feedback Refinement for HSI-LiDAR Open-Set Classification},
journal = {Remote Sensing},
year = {2026},
volume = {18},
number = {12},
pages = {1963},
doi = {10.3390/rs18121963}
}
EUVIMED