A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention
IEEE Geoscience and Remote Sensing Letters · 2024 · 21 · 1-5
DOI: 10.1109/lgrs.2024.3397993 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Geoscience and Remote Sensing Letters, 2024. Verfasst von Ming Zhao und 2 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 16 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Shanghai Municipal Natural Science Foundation.
Abstract (Originalsprache)
Registration of multimodal image pairs is a fundamental task in many remote sensing applications. To achieve accurate and low-cost remote sensing image registration, we propose a semi-supervised image registration framework based on multimodal cross-attention, which consists of the encoder for feature extraction, multimodal cross-attention module, and detection/descriptor decoders. We adopt positional encoding for feature maps to enhance the features with spatial contexts, especially for remote sensing images with large geometrical deformations. To learn common features that are independent of modalities between multimodal images, we proposed a multimodal cross-attention module to extract cross-modal features, which helps the detectors to extract more reliable matching keypoints. The network is trained in a semi-supervised manner, which requires only a small dataset of incompletely labeled images. To learn reliable keypoints from image pairs with inconsistent intensity and geometrical deformations, we randomly establish different geometrical mappings for the multimodal image pairs during training and then enrich the keypoint labels by continuously adding reliable keypoints extracted by the detection decoder in each epoch. Experimental results show that the proposed method achieves more comprehensive and accurate registration than the state-of-the-art methods for multimodal remote sensing images.
Förderung
- National Natural Science Foundation of China
- Shanghai Municipal Natural Science Foundation
Zitieren
APA
Zhao, M., Liu, J., & Wu, Y. (2024). A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention. IEEE Geoscience and Remote Sensing Letters, 21, 1-5. https://doi.org/10.1109/lgrs.2024.3397993
Vancouver
Zhao M, Liu J, Wu Y. A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention. IEEE Geoscience and Remote Sensing Letters. 2024;21:1-5. doi:10.1109/lgrs.2024.3397993
BibTeX
@article{Zhao2024,
author = {Zhao, Ming and Liu, Jingyi and Wu, Yan},
title = {A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention},
journal = {IEEE Geoscience and Remote Sensing Letters},
year = {2024},
volume = {21},
pages = {1--5},
doi = {10.1109/lgrs.2024.3397993}
}
EUVIMED