Mosaic Pattern Excavation Transformer for Spectral Imaging
IEEE Transactions on Image Processing · 2025 · 34 · 7817-7829
DOI: 10.1109/tip.2025.3633159 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Yaohang Wu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 61 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Chongqing Municipality, Hunan Natural Science Foundation for Outstanding Young Scholars.
Abstract (Originalsprache)
Single spectral image demosaicing for multispectral filter array (MSFA) is an essential task in spectral imaging, aiming to recover a mosaic-free spectral image from its mosaic raw counterpart. Existing deep learning-based methods typically improve the reconstruction performance by indiscriminately stacking CNN-based blocks, failing to effectively handle the intertwined spatio-spectral correlations caused by spatial sub-sampling and spectral aliasing. In this paper, we propose Mosaic Pattern Excavation Transformer (MPEFormer) to achieve better reconstruction by effectively modelling the intertwined spatio-spectral correlations. Specifically, the proposed three-branch model integrates low-frequency information, edge information, and fine high-frequency details essential for spectral image reconstruction, with the third branch serving as the core component. In this branch, we design the Dual Fusion Self-attention Block (DFSAB) and the Mosaic Pattern-guided Spectral Modulation Module (MPSM). DFSAB incorporates the Mosaic Pattern Excavation Self-attention (MPESA) mechanism, which effectively captures non-local spatio-spectral correlations induced by the MSFA pattern distributed across the whole image, thereby enhancing the expressive capability of the model. By dynamically integrating various MSFA pattern-related dependencies, MPSM enables adaptive recalibration of spectral information. Extensive experimental results demonstrate the effectiveness of our MPEFormer, highlighting its greater potential over the state-of-the-art MSFA demosaicing methods. The code will be uploaded at https://github.com/Matsuri247/MPEFormer.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Chongqing Municipality
- Hunan Natural Science Foundation for Outstanding Young Scholars
- Huxiang Project for Young Talents
- Fujian Provincial Natural Science Foundation of China
- 2024 Open Project Fund of the National Key Laboratory of Electromagnetic Space Security
- State Key Laboratory of Ocean Engineering
- State Key Laboratory of Spatial Datum
Zitieren
APA
Wu, Y., Liu, J., Dian, R., Li, S., & Yang, Y. (2025). Mosaic Pattern Excavation Transformer for Spectral Imaging. IEEE Transactions on Image Processing, 34, 7817-7829. https://doi.org/10.1109/tip.2025.3633159
Vancouver
Wu Y, Liu J, Dian R, Li S, Yang Y. Mosaic Pattern Excavation Transformer for Spectral Imaging. IEEE Transactions on Image Processing. 2025;34:7817-7829. doi:10.1109/tip.2025.3633159
BibTeX
@article{Wu2025,
author = {Wu, Yaohang and Liu, Jinyang and Dian, Renwei and Li, Shutao and Yang, Yining},
title = {Mosaic Pattern Excavation Transformer for Spectral Imaging},
journal = {IEEE Transactions on Image Processing},
year = {2025},
volume = {34},
pages = {7817--7829},
doi = {10.1109/tip.2025.3633159}
}
EUVIMED