EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Drone-Based Visible–Thermal Object Detection with Transformers and Prompt Tuning

Rui Chen, Dongdong Li, Zhinan Gao, Yangliu Kuai, Chengyuan Wang

Drones · 01.09.2024 · 8 (9) · 451

DOI: 10.3390/drones8090451 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Drones, online erschienen am 01.09.2024. Verfasst von Rui Chen und 4 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China (NSFC), scientific research project of National University of Defense Technology.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The use of unmanned aerial vehicles (UAVs) for visible–thermal object detection has emerged as a powerful technique to improve accuracy and resilience in challenging contexts, including dim lighting and severe weather conditions. However, most existing research relies on Convolutional Neural Network (CNN) frameworks, limiting the application of the Transformer’s attention mechanism to mere fusion modules and neglecting its potential for comprehensive global feature modeling. In response to this limitation, this study introduces an innovative dual-modal object detection framework called Visual Prompt multi-modal Detection (VIP-Det) that harnesses the Transformer architecture as the primary feature extractor and integrates vision prompts for refined feature fusion. Our approach begins with the training of a single-modal baseline model to solidify robust model representations, which is then refined through fine-tuning that incorporates additional modal data and prompts. Tests on the DroneVehicle dataset show that our algorithm achieves remarkable accuracy, outperforming comparable Transformer-based methods. These findings indicate that our proposed methodology marks a significant advancement in the realm of UAV-based object detection, holding significant promise for enhancing autonomous surveillance and monitoring capabilities in varied and challenging environments.

Förderung

  • National Natural Science Foundation of China (NSFC)
  • scientific research project of National University of Defense Technology

Zitieren

APA

Chen, R., Li, D., Gao, Z., Kuai, Y., & Wang, C. (2024). Drone-Based Visible–Thermal Object Detection with Transformers and Prompt Tuning. Drones, 8(9), 451. https://doi.org/10.3390/drones8090451

Vancouver

Chen R, Li D, Gao Z, Kuai Y, Wang C. Drone-Based Visible–Thermal Object Detection with Transformers and Prompt Tuning. Drones. 2024;8(9):451. doi:10.3390/drones8090451

BibTeX
@article{Chen2024,
  author = {Chen, Rui and Li, Dongdong and Gao, Zhinan and Kuai, Yangliu and Wang, Chengyuan},
  title = {Drone-Based Visible–Thermal Object Detection with Transformers and Prompt Tuning},
  journal = {Drones},
  year = {2024},
  volume = {8},
  number = {9},
  pages = {451},
  doi = {10.3390/drones8090451}
}
Kostenlos starten →