EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

DMDiff: A Dual-Branch Multimodal Conditional Guided Diffusion Model for Cloud Removal Through SAR-Optical Data Fusion

Wenjuan Zhang, Junlin Mei, Yuxi Wang

Remote Sensing · 09.03.2025 · 17 (6) · 965

DOI: 10.3390/rs17060965 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 09.03.2025. Verfasst von Wenjuan Zhang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Science and Disruptive Technology Program, AIRCAS.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Optical remote sensing images, as a significant data source for Earth observation, are often impacted by cloud cover, which severely limits their widespread application in Earth sciences. Synthetic aperture radar (SAR), with its all-weather, all-day observation capabilities, serves as a valuable auxiliary data source for cloud removal (CR) tasks. Despite substantial progress in deep learning (DL)-based CR methods utilizing SAR data in recent years, challenges remain in preserving fine texture details and maintaining image visual authenticity. To address these limitations, this study proposes a novel diffusion-based CR method called the Dual-branch Multimodal Conditional Guided Diffusion Model (DMDiff). Considering the intrinsic differences in data characteristics between SAR and optical images, we design a dual-branch feature extraction architecture to enable adaptive feature extraction based on the characteristics of the data. Then, a cross-attention mechanism is employed to achieve deep fusion of the multimodal feature extracted above, effectively guiding the progressive diffusion process to restore cloud-covered regions in optical images. Furthermore, we propose an image adaptive prediction (IAP) strategy within the diffusion model, specifically tailored to the characteristics of remote sensing data, which achieves a nearly 20 dB improvement in PSNR compared to the traditional noise prediction (NP) strategy. Extensive experiments on the airborne, WHU-OPT-SAR, and LuojiaSET-OSFCR datasets demonstrate that DMDiff outperforms SOTA methods in terms of both signal fidelity and visual perceptual quality. Specifically, on the LuojiaSET-OSFCR dataset, our method achieves a remarkable 17% reduction in the FID metric over the second-best method, while also yielding significant enhancements in quality assessment metrics such as PSNR and SSIM.

Förderung

  • National Natural Science Foundation of China
  • Science and Disruptive Technology Program, AIRCAS

Zitieren

APA

Zhang, W., Mei, J., & Wang, Y. (2025). DMDiff: A Dual-Branch Multimodal Conditional Guided Diffusion Model for Cloud Removal Through SAR-Optical Data Fusion. Remote Sensing, 17(6), 965. https://doi.org/10.3390/rs17060965

Vancouver

Zhang W, Mei J, Wang Y. DMDiff: A Dual-Branch Multimodal Conditional Guided Diffusion Model for Cloud Removal Through SAR-Optical Data Fusion. Remote Sensing. 2025;17(6):965. doi:10.3390/rs17060965

BibTeX
@article{Zhang2025,
  author = {Zhang, Wenjuan and Mei, Junlin and Wang, Yuxi},
  title = {DMDiff: A Dual-Branch Multimodal Conditional Guided Diffusion Model for Cloud Removal Through SAR-Optical Data Fusion},
  journal = {Remote Sensing},
  year = {2025},
  volume = {17},
  number = {6},
  pages = {965},
  doi = {10.3390/rs17060965}
}
Kostenlos starten →