Towards a Flexible Semantic Guided Model for Single Image Enhancement and Restoration
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 9921-9939
DOI: 10.1109/tpami.2024.3432308 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von Yuhui Wu und 8 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 87 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Sichuan Science and Technology Program, New Cornerstone Science Foundation.
Abstract (Originalsprache)
Low-light image enhancement (LLIE) investigates how to improve the brightness of an image captured in illumination-insufficient environments. The majority of existing methods enhance low-light images in a global and uniform manner, without taking into account the semantic information of different regions. Consequently, a network may easily deviate from the original color of local regions. To address this issue, we propose a semantic-aware knowledge-guided framework (SKF) that can assist a low-light enhancement model in learning rich and diverse priors encapsulated in a semantic segmentation model. We concentrate on incorporating semantic knowledge from three key aspects: a semantic-aware embedding module that adaptively integrates semantic priors in feature representation space, a semantic-guided color histogram loss that preserves color consistency of various instances, and a semantic-guided adversarial loss that produces more natural textures by semantic priors. Our SKF is appealing in acting as a general framework in the LLIE task. We further present a refined framework SKF++ with two new techniques: (a) Extra convolutional branch for intra-class illumination and color recovery through extracting local information and (b) Equalization-based histogram transformation for contrast enhancement and high dynamic range adjustment. Extensive experiments on various benchmarks of LLIE task and other image processing tasks show that models equipped with the SKF/SKF++ significantly outperform the baselines and our SKF/SKF++ generalizes to different models and scenes well. Besides, the potential benefits of our method in face detection and semantic segmentation in low-light conditions are discussed.
Förderung
- National Natural Science Foundation of China
- Sichuan Science and Technology Program
- New Cornerstone Science Foundation
Zitieren
APA
Wu, Y., Wang, G., Liu, S., Yang, Y., Li, W., Tang, X., Gu, S., Li, C., & Shen, H. T. (2024). Towards a Flexible Semantic Guided Model for Single Image Enhancement and Restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 9921-9939. https://doi.org/10.1109/tpami.2024.3432308
Vancouver
Wu Y, Wang G, Liu S, Yang Y, Li W, Tang X, et al. Towards a Flexible Semantic Guided Model for Single Image Enhancement and Restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):9921-9939. doi:10.1109/tpami.2024.3432308
BibTeX
@article{Wu2024,
author = {Wu, Yuhui and Wang, Guoqing and Liu, Shaochong and Yang, Yang and Li, Wei and Tang, Xiongxin and Gu, Shuhang and Li, Chongyi and Shen, Heng Tao},
title = {Towards a Flexible Semantic Guided Model for Single Image Enhancement and Restoration},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {12},
pages = {9921--9939},
doi = {10.1109/tpami.2024.3432308}
}
EUVIMED