EUVIMEDEUVIMED entdecken
Crossref

Scale-Semantic Joint Decoupling Network for Image-Text Retrieval in Remote Sensing

Chengyu Zheng, Ning Song, Ruoyu Zhang, Lei Huang, Zhiqiang Wei, Jie Nie

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.01.2024 · 20 (1) · 1-20

DOI: 10.1145/3603628 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 24.08.2023. Verfasst von Chengyu Zheng und 5 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 48 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, National Key Research and Development Program of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Image-text retrieval in remote sensing aims to provide flexible information for data analysis and application. In recent years, state-of-the-art methods are dedicated to “scale decoupling” and “semantic decoupling” strategies to further enhance the capability of representation. However, these previous approaches focus on either the disentangling scale or semantics but ignore merging these two ideas in a union model, which extremely limits the performance of cross-modal retrieval models. To address these issues, we propose a novel Scale-Semantic Joint Decoupling Network (SSJDN) for remote sensing image-text retrieval. Specifically, we design the Bidirectional Scale Decoupling (BSD) module, which exploits Salience Extraction Map (SEM) and Salience Suppression Map (SSM) units to adaptively extract potential features and suppress cumbersome features at other scales in a bidirectional pattern to yield different scale clues. Besides, we design the Label-supervised Semantic Decoupling (LSD) module by leveraging the category semantic labels as prior knowledge to supervise images and texts probing significant semantic-related information. Finally, we design a Semantic-guided Triple Loss (STL), which adaptively generates a constant to adjust the loss function to improve the probability of matching the same semantic image and text and shorten the convergence time of the retrieval model. Our proposed SSJDN outperforms state-of-the-art approaches in numerical experiments conducted on four benchmark remote sensing datasets.

Förderung

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities
  • National Key Research and Development Program of China

Zitieren

APA

Zheng, C., Song, N., Zhang, R., Huang, L., Wei, Z., & Nie, J. (2024). Scale-Semantic Joint Decoupling Network for Image-Text Retrieval in Remote Sensing. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(1), 1-20. https://doi.org/10.1145/3603628

Vancouver

Zheng C, Song N, Zhang R, Huang L, Wei Z, Nie J. Scale-Semantic Joint Decoupling Network for Image-Text Retrieval in Remote Sensing. ACM Transactions on Multimedia Computing, Communications, and Applications. 2024;20(1):1-20. doi:10.1145/3603628

BibTeX
@article{Zheng2024,
  author = {Zheng, Chengyu and Song, Ning and Zhang, Ruoyu and Huang, Lei and Wei, Zhiqiang and Nie, Jie},
  title = {Scale-Semantic Joint Decoupling Network for Image-Text Retrieval in Remote Sensing},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2024},
  volume = {20},
  number = {1},
  pages = {1--20},
  doi = {10.1145/3603628}
}
Kostenlos starten →