EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Cross-Modality Data Augmentation for Aerial Object Detection with Representation Learning

Chiheng Wei, Lianfa Bai, Xiaoyu Chen, Jing Han

Remote Sensing · 12.12.2024 · 16 (24) · 4649

DOI: 10.3390/rs16244649 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 12.12.2024. Verfasst von Chiheng Wei und 3 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Jiangsu Provincial Key Research and Development Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Data augmentation methods offer a cost-effective and efficient alternative to the acquisition of additional data, significantly enhancing data diversity and model generalization, making them particularly favored in object detection tasks. However, existing data augmentation techniques primarily focus on the visible spectrum and are directly applied to RGB-T object detection tasks, overlooking the inherent differences in image data between the two tasks. Visible images capture rich color and texture information during the daytime, while infrared images are capable of imaging under low-light complex scenarios during the nighttime. By integrating image information from both modalities, their complementary characteristics can be exploited to improve the overall effectiveness of data augmentation methods. To address this, we propose a cross-modality data augmentation method tailored for RGB-T object detection, leveraging masked image modeling within representation learning. Specifically, we focus on the temporal consistency of infrared images and combine them with visible images under varying lighting conditions for joint data augmentation, thereby enhancing the realism of the augmented images. Utilizing the masked image modeling method, we reconstruct images by integrating multimodal features, achieving cross-modality data augmentation in feature space. Additionally, we investigate the differences and complementarities between data augmentation methods in data space and feature space. Building upon existing theoretical foundations, we propose an integrative framework that combines these methods for improved augmentation effectiveness. Furthermore, we address the slow convergence observed with the existing Mosaic method in aerial imagery by introducing a multi-scale training strategy and proposing a full-scale Mosaic method as a complement. This optimization significantly accelerates network convergence. The experimental results validate the effectiveness of our proposed method and highlight its potential for further advancements in cross-modality object detection tasks.

Förderung

  • National Natural Science Foundation of China
  • Jiangsu Provincial Key Research and Development Program

Zitieren

APA

Wei, C., Bai, L., Chen, X., & Han, J. (2024). Cross-Modality Data Augmentation for Aerial Object Detection with Representation Learning. Remote Sensing, 16(24), 4649. https://doi.org/10.3390/rs16244649

Vancouver

Wei C, Bai L, Chen X, Han J. Cross-Modality Data Augmentation for Aerial Object Detection with Representation Learning. Remote Sensing. 2024;16(24):4649. doi:10.3390/rs16244649

BibTeX
@article{Wei2024,
  author = {Wei, Chiheng and Bai, Lianfa and Chen, Xiaoyu and Han, Jing},
  title = {Cross-Modality Data Augmentation for Aerial Object Detection with Representation Learning},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {24},
  pages = {4649},
  doi = {10.3390/rs16244649}
}
Kostenlos starten →