EUVIMEDEUVIMED entdecken
Crossref

A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention

Ming Zhao, Jingyi Liu, Yan Wu

IEEE Geoscience and Remote Sensing Letters · 2024 · 21 · 1-5

DOI: 10.1109/lgrs.2024.3397993 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Geoscience and Remote Sensing Letters, 2024. Verfasst von Ming Zhao und 2 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 16 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Shanghai Municipal Natural Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Registration of multimodal image pairs is a fundamental task in many remote sensing applications. To achieve accurate and low-cost remote sensing image registration, we propose a semi-supervised image registration framework based on multimodal cross-attention, which consists of the encoder for feature extraction, multimodal cross-attention module, and detection/descriptor decoders. We adopt positional encoding for feature maps to enhance the features with spatial contexts, especially for remote sensing images with large geometrical deformations. To learn common features that are independent of modalities between multimodal images, we proposed a multimodal cross-attention module to extract cross-modal features, which helps the detectors to extract more reliable matching keypoints. The network is trained in a semi-supervised manner, which requires only a small dataset of incompletely labeled images. To learn reliable keypoints from image pairs with inconsistent intensity and geometrical deformations, we randomly establish different geometrical mappings for the multimodal image pairs during training and then enrich the keypoint labels by continuously adding reliable keypoints extracted by the detection decoder in each epoch. Experimental results show that the proposed method achieves more comprehensive and accurate registration than the state-of-the-art methods for multimodal remote sensing images.

Förderung

  • National Natural Science Foundation of China
  • Shanghai Municipal Natural Science Foundation

Zitieren

APA

Zhao, M., Liu, J., & Wu, Y. (2024). A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention. IEEE Geoscience and Remote Sensing Letters, 21, 1-5. https://doi.org/10.1109/lgrs.2024.3397993

Vancouver

Zhao M, Liu J, Wu Y. A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention. IEEE Geoscience and Remote Sensing Letters. 2024;21:1-5. doi:10.1109/lgrs.2024.3397993

BibTeX
@article{Zhao2024,
  author = {Zhao, Ming and Liu, Jingyi and Wu, Yan},
  title = {A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention},
  journal = {IEEE Geoscience and Remote Sensing Letters},
  year = {2024},
  volume = {21},
  pages = {1--5},
  doi = {10.1109/lgrs.2024.3397993}
}
Kostenlos starten →