EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Multimodal Image Registration Method for UAV Visual Navigation Based on Feature Fusion and Transformers

Ruofei He, Shuangxing Long, Wei Sun, Hongjuan Liu

Drones · 07.11.2024 · 8 (11) · 651

DOI: 10.3390/drones8110651 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Drones, online erschienen am 07.11.2024. Verfasst von Ruofei He und 3 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Shaanxi Key R&D Plan Key Industry Innovation Chain Project, China College Innovation Fund of Production, Education, and Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Using images captured by drone cameras and comparing them with known Google satellite maps to obtain the current location of the drone is an important way of UAV navigation in GPS-denied environments. But, due to inherent modality differences and significant geometric deformations, cross-modal image registration is challenging. This paper proposes a CNN-Transformer hybrid network model for feature detection and feature matching. ResNet50 is used as the backbone network for feature extraction. An improved feature fusion module is used to fuse feature maps from different levels, and then a Transformer encoder–decoder structure is used for feature matching to obtain preliminary correspondences. Finally, a geometric outlier removal method (GSM) is used to eliminate mismatched points based on the geometric similarity of inliers, resulting in more robust correspondences. Qualitative and quantitative experiments were conducted on multimodal image datasets captured by UAVs; the correct matching rate was improved by 52%, 21%, and 15%, respectively, and the error was reduced by 36% compared to the 3MRS algorithm. A total of 56 experiments were conducted in actual scenarios, with a localization success rate of 91.1%, and the RMSE of UAV positioning was 4.6 m.

Förderung

  • National Natural Science Foundation of China
  • Shaanxi Key R&D Plan Key Industry Innovation Chain Project
  • China College Innovation Fund of Production, Education, and Research
  • Xi’an Science and Technology Plan Project

Zitieren

APA

He, R., Long, S., Sun, W., & Liu, H. (2024). A Multimodal Image Registration Method for UAV Visual Navigation Based on Feature Fusion and Transformers. Drones, 8(11), 651. https://doi.org/10.3390/drones8110651

Vancouver

He R, Long S, Sun W, Liu H. A Multimodal Image Registration Method for UAV Visual Navigation Based on Feature Fusion and Transformers. Drones. 2024;8(11):651. doi:10.3390/drones8110651

BibTeX
@article{He2024,
  author = {He, Ruofei and Long, Shuangxing and Sun, Wei and Liu, Hongjuan},
  title = {A Multimodal Image Registration Method for UAV Visual Navigation Based on Feature Fusion and Transformers},
  journal = {Drones},
  year = {2024},
  volume = {8},
  number = {11},
  pages = {651},
  doi = {10.3390/drones8110651}
}
Kostenlos starten →