Diff-Restorer: Unleashing Visual Prompts for Diffusion-Based Universal Image Restoration
IEEE Transactions on Circuits and Systems for Video Technology · 2026-04 · 36 (4) · 4259-4274
DOI: 10.1109/tcsvt.2025.3629686 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-04. Verfasst von Yuhong Zhang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 94 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Science and Technology Commission of Shanghai Municipality, Fundamental Research Funds for the Central Universities, Shanghai Key Laboratory of Digital Media Processing and Transmission.
Abstract (Originalsprache)
Image restoration aims to recover high-quality images from degraded observations, yet real-world degradations are complex, coupled, and difficult to model. Existing task-specific methods struggle to generalize beyond predefined degradation types, while recent all-in-one or prompt-based methods still face three key challenges: 1) they rely on task-specific training or fixed prompt pools, limiting adaptability to real-world and mixed degradations; 2) human-instruction or implicit-prompt mechanisms make them difficult to use in practice; and 3) they often fail to balance structural fidelity and perceptual realism. To address these issues, we propose Diff-Restorer, a diffusion-based universal image restoration framework that unifies diverse degradation handling within a single model. Diff-Restorer adaptively extracts decoupled visual prompts from a visual-language model (CLIP), including clear semantic and degradation embeddings. The clear semantic embeddings serve as content prompts to guide the diffusion model for generation, improving perceptual quality. The degradation embeddings as the task identifier modulate the Image-guided Control Module to generate structure control, ensuring faithfulness. Furthermore, we design a Task-aware Decoder to perform structural correction and convert the latent code to the pixel domain. Extensive experiments on various single, real-world, and mixed degradation tasks show that Diff-Restorer outperforms state-of-the-art methods in terms of generality, realism, and fidelity.
Förderung
- National Natural Science Foundation of China
- Science and Technology Commission of Shanghai Municipality
- Fundamental Research Funds for the Central Universities, Shanghai Key Laboratory of Digital Media Processing and Transmission
- Higher Education Discipline Innovation Project
Zitieren
APA
Zhang, Y., Zhang, H., Chai, X., Cheng, Z., Xie, R., Song, L., & Zhang, W. (2026). Diff-Restorer: Unleashing Visual Prompts for Diffusion-Based Universal Image Restoration. IEEE Transactions on Circuits and Systems for Video Technology, 36(4), 4259-4274. https://doi.org/10.1109/tcsvt.2025.3629686
Vancouver
Zhang Y, Zhang H, Chai X, Cheng Z, Xie R, Song L, et al. Diff-Restorer: Unleashing Visual Prompts for Diffusion-Based Universal Image Restoration. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(4):4259-4274. doi:10.1109/tcsvt.2025.3629686
BibTeX
@article{Zhang2026,
author = {Zhang, Yuhong and Zhang, Hengsheng and Chai, Xinning and Cheng, Zhengxue and Xie, Rong and Song, Li and Zhang, Wenjun},
title = {Diff-Restorer: Unleashing Visual Prompts for Diffusion-Based Universal Image Restoration},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {4},
pages = {4259--4274},
doi = {10.1109/tcsvt.2025.3629686}
}
EUVIMED