Plug-and-Play General Image Registration for Misaligned Multi-Modal Image Fusion
IEEE Transactions on Circuits and Systems for Video Technology · 2025-10 · 35 (10) · 10017-10031
DOI: 10.1109/tcsvt.2025.3570530 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-10. Verfasst von Tianheng Zheng und 4 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 67 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Sichuan Province, Cooperation Science and Technology Project of Sichuan University and Dazhou City.
Abstract (Originalsprache)
Contemporary works in multi-modal image fusion often excessively rely on aligned source images, resulting in limited practicality when encountering misaligned data. However, there is still a significant gap in developing effective multi-modal image registration methods to address this problem. Moreover, existing multi-modal image registration models are largely restricted to specific types of multi-modal data, lacking a general model applicable to diverse multi-modal data types. To address th above issues, this study introduces a novel method named PGMR, which stands as the first plug-and-play general multi-modal image registration model. PGMR comprises three components: Modality Prompt Module (MPM), Universal Registration Framework (URF), and Detail Enhancement Module (DEM). URF serves as the fundamental registration framework, handling both rigid and non-rigid deformations to achieve basic multi-modal image registration. MPM, one core component of this paper, is embedded within URF. Leveraging prompt learning, MPM dynamically integrates modality prompts into the intermediate output of URF, not only alleviating modality discrepancies but also promoting the ability of the registration model across various multi-modal data types. DEM is a detail enhancement module for multi-modal image registration. It can enrich the details of registration results, thereby enhancing the effectiveness of subsequent tasks. We evaluate the performance of PGMR on four multi-modal types and extensive experiments validate the feasibility of PGMR, demonstrating the superiority of our method compared to state-of-the-art alternatives. The code will be available at https://github.com/stwts/PGMR
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Sichuan Province
- Cooperation Science and Technology Project of Sichuan University and Dazhou City
- Young Faculty Technology Innovation Capacity Enhancement Program of Sichuan University
- Open Project of Anhui Provincial Key Laboratory of Multimodal Cognitive Computation
- Fundamental Research Funds for the Central Universities
Zitieren
APA
Zheng, T., Dong, G., Zhang, P., He, X., & Ren, C. (2025). Plug-and-Play General Image Registration for Misaligned Multi-Modal Image Fusion. IEEE Transactions on Circuits and Systems for Video Technology, 35(10), 10017-10031. https://doi.org/10.1109/tcsvt.2025.3570530
Vancouver
Zheng T, Dong G, Zhang P, He X, Ren C. Plug-and-Play General Image Registration for Misaligned Multi-Modal Image Fusion. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(10):10017-10031. doi:10.1109/tcsvt.2025.3570530
BibTeX
@article{Zheng2025,
author = {Zheng, Tianheng and Dong, Guanglu and Zhang, Pingping and He, Xiaohai and Ren, Chao},
title = {Plug-and-Play General Image Registration for Misaligned Multi-Modal Image Fusion},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2025},
volume = {35},
number = {10},
pages = {10017--10031},
doi = {10.1109/tcsvt.2025.3570530}
}
EUVIMED