Image De-Raining Transformer
IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.11.2023 · 45 (11) · 12978-12995
DOI: 10.1109/tpami.2022.3183612 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.11.2023. Verfasst von Jie Xiao und 4 weiteren Autor:innen. Die Arbeit wurde bisher 399-mal zitiert (Crossref) und nennt 121 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China, University Synergy Innovation Program of Anhui Province.
Abstract (Originalsprache)
Existing deep learning based de-raining approaches have resorted to the convolutional architectures. However, the intrinsic limitations of convolution, including local receptive fields and independence of input content, hinder the model's ability to capture long-range and complicated rainy artifacts. To overcome these limitations, we propose an effective and efficient transformer-based architecture for the image de-raining. First, we introduce general priors of vision tasks, i.e., locality and hierarchy, into the network architecture so that our model can achieve excellent de-raining performance without costly pre-training. Second, since the geometric appearance of rainy artifacts is complicated and of significant variance in space, it is essential for de-raining models to extract both local and non-local features. Therefore, we design the complementary window-based transformer and spatial transformer to enhance locality while capturing long-range dependencies. Besides, to compensate for the positional blindness of self-attention, we establish a separate representative space for modeling positional relationship, and design a new relative position enhanced multi-head self-attention. In this way, our model enjoys powerful abilities to capture dependencies from both content and position, so as to achieve better image content recovery while removing rainy artifacts. Experiments substantiate that our approach attains more appealing results than state-of-the-art methods quantitatively and qualitatively.
Förderung
- National Key R&D Program of China
- National Natural Science Foundation of China
- University Synergy Innovation Program of Anhui Province
- Fundamental Research Funds for the Central Universities
- USTC Research Funds of the Double First-Class Initiative
Zitieren
APA
Xiao, J., Fu, X., Liu, A., Wu, F., & Zha, Z. J. (2023). Image De-Raining Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11), 12978-12995. https://doi.org/10.1109/tpami.2022.3183612
Vancouver
Xiao J, Fu X, Liu A, Wu F, Zha ZJ. Image De-Raining Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(11):12978-12995. doi:10.1109/tpami.2022.3183612
BibTeX
@article{Xiao2023,
author = {Xiao, Jie and Fu, Xueyang and Liu, Aiping and Wu, Feng and Zha, Zheng-Jun},
title = {Image De-Raining Transformer},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2023},
volume = {45},
number = {11},
pages = {12978--12995},
doi = {10.1109/tpami.2022.3183612}
}
EUVIMED