EUVIMEDEUVIMED entdecken
Crossref

Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models

Fei Shen, Hu Ye, Sibo Liu, Jun Zhang, Cong Wang, Xiao Han, Yang Wei

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (7) · 6785-6794

DOI: 10.1609/aaai.v39i7.32728 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Fei Shen und 6 weiteren Autor:innen. Die Arbeit wurde bisher 38-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent research showcases the considerable potential of conditional diffusion models for generating consistent stories. However, current methods, which primarily generate stories in a caption-dependent manner, often overlook the importance of contextual consistency and the relevance of frames during sequential generation. To address this, we propose a novel Rich-contextual Conditional Diffusion Models (RCDMs), a two-stage approach designed to enhance story generation's semantic consistency and temporal consistency. Specifically, in the first stage, the frame-prior transformer diffusion model is presented to predict the frame semantic embedding of the unknown clip by aligning the semantic correlations between the captions and frames of the known clip. The second stage establishes a robust model with rich contextual conditions, including reference images of the known clip, the predicted frame semantic embedding of the unknown clip, and text embeddings of all captions. By jointly injecting these rich contextual conditions at the image and feature levels, RCDMs can generate semantic and temporal consistency stories. Moreover, RCDMs can generate consistent stories with a single forward inference compared to autoregressive models. Our qualitative and quantitative results demonstrate that our proposed RCDMs outperform in challenging scenarios.

Zitieren

APA

Shen, F., Ye, H., Liu, S., Zhang, J., Wang, C., Han, X., & Wei, Y. (2025). Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence, 39(7), 6785-6794. https://doi.org/10.1609/aaai.v39i7.32728

Vancouver

Shen F, Ye H, Liu S, Zhang J, Wang C, Han X, et al. Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(7):6785-6794. doi:10.1609/aaai.v39i7.32728

BibTeX
@article{Shen2025,
  author = {Shen, Fei and Ye, Hu and Liu, Sibo and Zhang, Jun and Wang, Cong and Han, Xiao and Wei, Yang},
  title = {Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {7},
  pages = {6785--6794},
  doi = {10.1609/aaai.v39i7.32728}
}
Kostenlos starten →