Drone-Based Visible–Thermal Object Detection with Transformers and Prompt Tuning
Drones · 01.09.2024 · 8 (9) · 451
DOI: 10.3390/drones8090451 ↗Kurz gefasst
Fachartikel in der Zeitschrift Drones, online erschienen am 01.09.2024. Verfasst von Rui Chen und 4 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China (NSFC), scientific research project of National University of Defense Technology.
Abstract (Originalsprache)
The use of unmanned aerial vehicles (UAVs) for visible–thermal object detection has emerged as a powerful technique to improve accuracy and resilience in challenging contexts, including dim lighting and severe weather conditions. However, most existing research relies on Convolutional Neural Network (CNN) frameworks, limiting the application of the Transformer’s attention mechanism to mere fusion modules and neglecting its potential for comprehensive global feature modeling. In response to this limitation, this study introduces an innovative dual-modal object detection framework called Visual Prompt multi-modal Detection (VIP-Det) that harnesses the Transformer architecture as the primary feature extractor and integrates vision prompts for refined feature fusion. Our approach begins with the training of a single-modal baseline model to solidify robust model representations, which is then refined through fine-tuning that incorporates additional modal data and prompts. Tests on the DroneVehicle dataset show that our algorithm achieves remarkable accuracy, outperforming comparable Transformer-based methods. These findings indicate that our proposed methodology marks a significant advancement in the realm of UAV-based object detection, holding significant promise for enhancing autonomous surveillance and monitoring capabilities in varied and challenging environments.
Förderung
- National Natural Science Foundation of China (NSFC)
- scientific research project of National University of Defense Technology
Zitieren
APA
Chen, R., Li, D., Gao, Z., Kuai, Y., & Wang, C. (2024). Drone-Based Visible–Thermal Object Detection with Transformers and Prompt Tuning. Drones, 8(9), 451. https://doi.org/10.3390/drones8090451
Vancouver
Chen R, Li D, Gao Z, Kuai Y, Wang C. Drone-Based Visible–Thermal Object Detection with Transformers and Prompt Tuning. Drones. 2024;8(9):451. doi:10.3390/drones8090451
BibTeX
@article{Chen2024,
author = {Chen, Rui and Li, Dongdong and Gao, Zhinan and Kuai, Yangliu and Wang, Chengyuan},
title = {Drone-Based Visible–Thermal Object Detection with Transformers and Prompt Tuning},
journal = {Drones},
year = {2024},
volume = {8},
number = {9},
pages = {451},
doi = {10.3390/drones8090451}
}
EUVIMED