Learnable Nonlocal Self-Similarity of Deep Features for Image Denoising
SIAM Journal on Imaging Sciences · 31.03.2024 · 17 (1) · 441-475
DOI: 10.1137/22m1536996 ↗Kurz gefasst
Fachartikel in der Zeitschrift SIAM Journal on Imaging Sciences, online erschienen am 23.02.2024. Verfasst von Junying Meng und 2 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 50 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Beijing Municipality.
Abstract (Originalsprache)
High-dimensional deep features extracted by convolutional neural networks have nonlocal self-similarity. However, incorporating this nonlocal prior of deep features into deep network architectures with an interpretable variational framework is rarely explored. In this paper, we propose a learnable nonlocal self-similarity deep feature network for image denoising. Our method is motivated by the fact that the high-dimensional deep features obey a mixture probability distribution based on the Parzen–Rosenblatt window method. Then a regularizer with learnable nonlocal weights is proposed by considering the dual representation of the log-probability prior of the deep features. Specifically, the nonlocal weights are introduced as dual variables that can be learned by unrolling the associated numerical scheme. This leads to nonlocal modules (NLMs) in newly designed networks. Our method provides a statistical and variational interpretation for the nonlocal self-attention mechanism widely used in various networks. By adopting nonoverlapping window and region decomposition techniques, we can significantly reduce the computational complexity of nonlocal self-similarity, thus enabling parallel computation of the NLM. The solution to the proposed variational problem can be formulated as a learnable nonlocal self-similarity network for image denoising. This work offers a novel approach for constructing network structures that consider self-similarity and nonlocality. The improvements achieved by this method are predictable and partially controllable. Compared with several closely related denoising methods, the experimental results show the effectiveness of the proposed method in image denoising.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Beijing Municipality
Zitieren
APA
Meng, J., Wang, F., & Liu, J. (2024). Learnable Nonlocal Self-Similarity of Deep Features for Image Denoising. SIAM Journal on Imaging Sciences, 17(1), 441-475. https://doi.org/10.1137/22m1536996
Vancouver
Meng J, Wang F, Liu J. Learnable Nonlocal Self-Similarity of Deep Features for Image Denoising. SIAM Journal on Imaging Sciences. 2024;17(1):441-475. doi:10.1137/22m1536996
BibTeX
@article{Meng2024,
author = {Meng, Junying and Wang, Faqiang and Liu, Jun},
title = {Learnable Nonlocal Self-Similarity of Deep Features for Image Denoising},
journal = {SIAM Journal on Imaging Sciences},
year = {2024},
volume = {17},
number = {1},
pages = {441--475},
doi = {10.1137/22m1536996}
}
EUVIMED