FrDiff: Framelet-Based Conditional Diffusion Model for Multispectral and Panchromatic Image Fusion
IEEE Transactions on Multimedia · 2025 · 27 · 5989-6002
DOI: 10.1109/tmm.2025.3565985 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Multimedia, 2025. Verfasst von Junkang Zhang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 79 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China, AI-Empowered Research Paradigm Reform and Discipline Leap Plan.
Abstract (Originalsprache)
The process of fusing low-resolution multispectral (LRMS) and high-resolution panchromatic (PAN) imagery, commonly referred to as pansharpening, is intended to generate high-resolution multispectral (HRMS) imagery. Typically, most pre-existing pansharpening frameworks mainly emphasize the straightforward learning of the mapping relationship among PAN and LRMS images to HRMS images. However, a key limitation of these frameworks is their potential overemphasis on spatial information, particularly the enhancement of low-frequency components. As a result, such an oversight potentially hinders the model's ability to simultaneously restore both spectral and spatial details. To address this issue, we propose a novel pansharpening model based on the denoising diffusion probabilistic model (DDPM), dubbed FrDiff. Specifically, we build a framelet-based conditional diffusion model that leverages the generative power of diffusion models to produce more refine results. Different from conventional methods directly inferring HRMS images, our strategy is designed to project their framelet coefficients, utilizing the available PAN and LRMS images as resources. This approach enables the separation of high-frequency and low-frequency components through framelet transformation, which are subsequently recombined to create a novel set of conditional embeddings that feed into the diffusion process. At the same time, the powerful predictive power of the diffusion model is exploited to simultaneously recover the high-frequency and low-frequency components of the HRMS. Moreover, we introduce a framelet-oriented cross-attention module dedicated to honing spectral fidelity. This module is crucial for improving the spectral precision of the HRMS images, ensuring a balanced emphasis on both spatial and spectral enhancements. Quantitative and qualitative experiments on multiple benchmark datasets demonstrate that the proposed method achieves more robustness and high-quality results than other state-of-the-art pansharpening methods.
Förderung
- National Key R&D Program of China
- National Natural Science Foundation of China
- AI-Empowered Research Paradigm Reform and Discipline Leap Plan
- Open Research Fund of KLATASDS-MOE
- East China Normal University
Zitieren
APA
Zhang, J., Fang, F., Wang, T., Zhang, G., & Song, H. (2025). FrDiff: Framelet-Based Conditional Diffusion Model for Multispectral and Panchromatic Image Fusion. IEEE Transactions on Multimedia, 27, 5989-6002. https://doi.org/10.1109/tmm.2025.3565985
Vancouver
Zhang J, Fang F, Wang T, Zhang G, Song H. FrDiff: Framelet-Based Conditional Diffusion Model for Multispectral and Panchromatic Image Fusion. IEEE Transactions on Multimedia. 2025;27:5989-6002. doi:10.1109/tmm.2025.3565985
BibTeX
@article{Zhang2025,
author = {Zhang, Junkang and Fang, Faming and Wang, Tingting and Zhang, Guixu and Song, Haichuan},
title = {FrDiff: Framelet-Based Conditional Diffusion Model for Multispectral and Panchromatic Image Fusion},
journal = {IEEE Transactions on Multimedia},
year = {2025},
volume = {27},
pages = {5989--6002},
doi = {10.1109/tmm.2025.3565985}
}
EUVIMED