EUVIMEDEUVIMED entdecken
Crossref

StoryCrafter: Instance-Aligned Multi-Character Storytelling with Diffusion Policy Learning and Test-Time Reinforcement Learning

Ruihan Wu, Ruiqi Dong, Wenjing Pang, Chenjie Pan, Rui Zhang, Haishan Sun, Heng-yang Lu, Chenyou Fan

ACM Transactions on Intelligent Systems and Technology · 16.11.2026

DOI: 10.1145/3856814 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Intelligent Systems and Technology, online erschienen am 16.11.2026. Verfasst von Ruihan Wu und 7 weiteren Autor:innen. Die Arbeit nennt 56 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Open-ended visual storytelling poses a significant challenge for text-to-image models, which often fail to maintain narrative coherence and consistent character appearance across a sequence of images. To address this, we introduce StoryCrafter, a multi-character diffusion model that employs a novel instance-level cross-attention module. This module, trained with supervised fine-tuning, ensures precise alignment between text descriptions and characters, enabling consistent multi-character interactions throughout a story. Furthermore, we propose Direct-Diffusion Group Relative Policy Optimization, a novel reinforcement learning stage that uses story-aligned rewards to select optimal frames from a generated group, mitigating accumulated inconsistencies. We show that this group selection capability can be seamlessly integrated into inference time, effectively ensuring multi-frame style and character consistency, outperforming standard static greedy generation. We evaluate our approach through automated metrics, human assessments and vision-language model (VLM) judgment, measuring text-to-image alignment, style and character consistency, and fine-grained detail quality. Experiments on three benchmarks demonstrate that StoryCrafter outperforms existing methods, achieving 5.2% improvements in storytelling consistency on StorySalon dataset and 3.3%-10.8% in character accuracy on Flintstones-SV and Pororo-SV datasets, and also outperforming baselines in both human and VLM evaluations. The TTRL further brings consistent improvements in human evaluation on story visualization and continuation tasks, improving the average rank from 2.79 to 1.80 and from 1.80 to 1.39, respectively.

Zitieren

APA

Wu, R., Dong, R., Pang, W., Pan, C., Zhang, R., Sun, H., Lu, H., & Fan, C. (2026). StoryCrafter: Instance-Aligned Multi-Character Storytelling with Diffusion Policy Learning and Test-Time Reinforcement Learning. ACM Transactions on Intelligent Systems and Technology. https://doi.org/10.1145/3856814

Vancouver

Wu R, Dong R, Pang W, Pan C, Zhang R, Sun H, et al. StoryCrafter: Instance-Aligned Multi-Character Storytelling with Diffusion Policy Learning and Test-Time Reinforcement Learning. ACM Transactions on Intelligent Systems and Technology. 2026. doi:10.1145/3856814

BibTeX
@article{Wu2026,
  author = {Wu, Ruihan and Dong, Ruiqi and Pang, Wenjing and Pan, Chenjie and Zhang, Rui and Sun, Haishan and Lu, Heng-yang and Fan, Chenyou},
  title = {StoryCrafter: Instance-Aligned Multi-Character Storytelling with Diffusion Policy Learning and Test-Time Reinforcement Learning},
  journal = {ACM Transactions on Intelligent Systems and Technology},
  year = {2026},
  doi = {10.1145/3856814}
}
Kostenlos starten →