MURF: Mutually Reinforcing Multi-Modal Image Registration and Fusion
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2023-10 · 45 (10) · 12148-12166
DOI: 10.1109/tpami.2023.3283682 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023-10. Verfasst von Han Xu und 2 weiteren Autor:innen. Die Arbeit wurde bisher 316-mal zitiert (Crossref) und nennt 62 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Key Research and Development Program of Hubei Province.
Abstract (Originalsprache)
Existing image fusion methods are typically limited to aligned source images and have to "tolerate" parallaxes when images are unaligned. Simultaneously, the large variances between different modalities pose a significant challenge for multi-modal image registration. This study proposes a novel method called MURF, where for the first time, image registration and fusion are mutually reinforced rather than being treated as separate issues. MURF leverages three modules: shared information extraction module (SIEM), multi-scale coarse registration module (MCRM), and fine registration and fusion module (F2M). The registration is carried out in a coarse-to-fine manner. During coarse registration, SIEM first transforms multi-modal images into mono-modal shared information to eliminate the modal variances. Then, MCRM progressively corrects the global rigid parallaxes. Subsequently, fine registration to repair local non-rigid offsets and image fusion are uniformly implemented in F2M. The fused image provides feedback to improve registration accuracy, and the improved registration result further improves the fusion result. For image fusion, rather than solely preserving the original source information in existing methods, we attempt to incorporate texture enhancement into image fusion. We test on four types of multi-modal data (RGB-IR, RGB-NIR, PET-MRI, and CT-MRI). Extensive registration and fusion results validate the superiority and universality of MURF.
Förderung
- National Natural Science Foundation of China
- Key Research and Development Program of Hubei Province
Zitieren
APA
Xu, H., Yuan, J., & Ma, J. (2023). MURF: Mutually Reinforcing Multi-Modal Image Registration and Fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10), 12148-12166. https://doi.org/10.1109/tpami.2023.3283682
Vancouver
Xu H, Yuan J, Ma J. MURF: Mutually Reinforcing Multi-Modal Image Registration and Fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(10):12148-12166. doi:10.1109/tpami.2023.3283682
BibTeX
@article{Xu2023,
author = {Xu, Han and Yuan, Jiteng and Ma, Jiayi},
title = {MURF: Mutually Reinforcing Multi-Modal Image Registration and Fusion},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2023},
volume = {45},
number = {10},
pages = {12148--12166},
doi = {10.1109/tpami.2023.3283682}
}
EUVIMED