Stage-Aware Robust Multimodal Prior Guidance for Diffusion-Based Image Super-Resolution
Electronics · 12.07.2026 · 15 (14) · 3059
DOI: 10.3390/electronics15143059 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 12.07.2026. Verfasst von Changyuan Wang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Guangdong-Macao Cooperation Science and Technology Project, Science and Technology Development Fund.
Abstract (Originalsprache)
Diffusion-based image super-resolution (SR) has recently achieved impressive perceptual quality by progressively generating plausible high-resolution details. However, its restoration performance still depends strongly on the reliability of the conditioning signal derived from degraded low-resolution inputs. Under severe or complex degradations, LR-derived conditions may become incomplete or ambiguous, leading the denoising trajectory toward visually plausible but input-inconsistent reconstructions. This work focuses on a central question: how to construct reliable multimodal prior guidance for a diffusion backbone that commonly adopts a hierarchical U-shaped architecture. To this end, we propose STMP-DiT, a stage-aware text-aligned multimodal prior-guided Diffusion Transformer for image super-resolution. From a multimodal data mining perspective, STMP-DiT aims to discover, align, and organize complementary semantic and structural priors from heterogeneous foundation-model representations. To improve the reliability of semantic guidance, STMP-DiT first aligns LLaVA-derived LR prompts with the frozen CLIP HR-image embedding space, producing visually grounded textual priors for restoration. These aligned textual priors are complemented by hierarchical DINO features, where deep features guide coarse semantic layout, intermediate features support structural recovery, and shallow features refine local edges and textures in the U-shaped DiT backbone. Rather than treating textual and visual priors as a single homogeneous condition, STMP-DiT assigns hierarchical DINO priors to different restoration stages according to their representational granularity. The fused condition is then injected through bounded feature modulation, enabling controlled stage-aware guidance while reducing redundant conditioning and improving the parameter efficiency of the conditioning modules. Experimental results on widely used SR benchmarks indicate promising improvements in perceptual quality and distributional realism with a compact trainable parameter scale, suggesting the value of reliable, stage-aware, and parameter-efficient multimodal prior integration for diffusion-based image super-resolution.
Förderung
- Guangdong-Macao Cooperation Science and Technology Project
- Science and Technology Development Fund
Zitieren
APA
Wang, C., Shi, Z., & Chen, L. (2026). Stage-Aware Robust Multimodal Prior Guidance for Diffusion-Based Image Super-Resolution. Electronics, 15(14), 3059. https://doi.org/10.3390/electronics15143059
Vancouver
Wang C, Shi Z, Chen L. Stage-Aware Robust Multimodal Prior Guidance for Diffusion-Based Image Super-Resolution. Electronics. 2026;15(14):3059. doi:10.3390/electronics15143059
BibTeX
@article{Wang2026,
author = {Wang, Changyuan and Shi, Zhaoyin and Chen, Long},
title = {Stage-Aware Robust Multimodal Prior Guidance for Diffusion-Based Image Super-Resolution},
journal = {Electronics},
year = {2026},
volume = {15},
number = {14},
pages = {3059},
doi = {10.3390/electronics15143059}
}
EUVIMED