EUVIMEDEUVIMED entdecken
Crossref

Plug-and-Play General Image Registration for Misaligned Multi-Modal Image Fusion

Tianheng Zheng, Guanglu Dong, Pingping Zhang, Xiaohai He, Chao Ren

IEEE Transactions on Circuits and Systems for Video Technology · 2025-10 · 35 (10) · 10017-10031

DOI: 10.1109/tcsvt.2025.3570530 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-10. Verfasst von Tianheng Zheng und 4 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 67 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Sichuan Province, Cooperation Science and Technology Project of Sichuan University and Dazhou City.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Contemporary works in multi-modal image fusion often excessively rely on aligned source images, resulting in limited practicality when encountering misaligned data. However, there is still a significant gap in developing effective multi-modal image registration methods to address this problem. Moreover, existing multi-modal image registration models are largely restricted to specific types of multi-modal data, lacking a general model applicable to diverse multi-modal data types. To address th above issues, this study introduces a novel method named PGMR, which stands as the first plug-and-play general multi-modal image registration model. PGMR comprises three components: Modality Prompt Module (MPM), Universal Registration Framework (URF), and Detail Enhancement Module (DEM). URF serves as the fundamental registration framework, handling both rigid and non-rigid deformations to achieve basic multi-modal image registration. MPM, one core component of this paper, is embedded within URF. Leveraging prompt learning, MPM dynamically integrates modality prompts into the intermediate output of URF, not only alleviating modality discrepancies but also promoting the ability of the registration model across various multi-modal data types. DEM is a detail enhancement module for multi-modal image registration. It can enrich the details of registration results, thereby enhancing the effectiveness of subsequent tasks. We evaluate the performance of PGMR on four multi-modal types and extensive experiments validate the feasibility of PGMR, demonstrating the superiority of our method compared to state-of-the-art alternatives. The code will be available at https://github.com/stwts/PGMR

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Sichuan Province
  • Cooperation Science and Technology Project of Sichuan University and Dazhou City
  • Young Faculty Technology Innovation Capacity Enhancement Program of Sichuan University
  • Open Project of Anhui Provincial Key Laboratory of Multimodal Cognitive Computation
  • Fundamental Research Funds for the Central Universities

Zitieren

APA

Zheng, T., Dong, G., Zhang, P., He, X., & Ren, C. (2025). Plug-and-Play General Image Registration for Misaligned Multi-Modal Image Fusion. IEEE Transactions on Circuits and Systems for Video Technology, 35(10), 10017-10031. https://doi.org/10.1109/tcsvt.2025.3570530

Vancouver

Zheng T, Dong G, Zhang P, He X, Ren C. Plug-and-Play General Image Registration for Misaligned Multi-Modal Image Fusion. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(10):10017-10031. doi:10.1109/tcsvt.2025.3570530

BibTeX
@article{Zheng2025,
  author = {Zheng, Tianheng and Dong, Guanglu and Zhang, Pingping and He, Xiaohai and Ren, Chao},
  title = {Plug-and-Play General Image Registration for Misaligned Multi-Modal Image Fusion},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {10},
  pages = {10017--10031},
  doi = {10.1109/tcsvt.2025.3570530}
}
Kostenlos starten →