StoryCrafter: Instance-Aligned Multi-Character Storytelling with Diffusion Policy Learning and Test-Time Reinforcement Learning
ACM Transactions on Intelligent Systems and Technology · 16.11.2026
DOI: 10.1145/3856814 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Intelligent Systems and Technology, online erschienen am 16.11.2026. Verfasst von Ruihan Wu und 7 weiteren Autor:innen. Die Arbeit nennt 56 Literaturangaben.
Abstract (Originalsprache)
Open-ended visual storytelling poses a significant challenge for text-to-image models, which often fail to maintain narrative coherence and consistent character appearance across a sequence of images. To address this, we introduce StoryCrafter, a multi-character diffusion model that employs a novel instance-level cross-attention module. This module, trained with supervised fine-tuning, ensures precise alignment between text descriptions and characters, enabling consistent multi-character interactions throughout a story. Furthermore, we propose Direct-Diffusion Group Relative Policy Optimization, a novel reinforcement learning stage that uses story-aligned rewards to select optimal frames from a generated group, mitigating accumulated inconsistencies. We show that this group selection capability can be seamlessly integrated into inference time, effectively ensuring multi-frame style and character consistency, outperforming standard static greedy generation. We evaluate our approach through automated metrics, human assessments and vision-language model (VLM) judgment, measuring text-to-image alignment, style and character consistency, and fine-grained detail quality. Experiments on three benchmarks demonstrate that StoryCrafter outperforms existing methods, achieving 5.2% improvements in storytelling consistency on StorySalon dataset and 3.3%-10.8% in character accuracy on Flintstones-SV and Pororo-SV datasets, and also outperforming baselines in both human and VLM evaluations. The TTRL further brings consistent improvements in human evaluation on story visualization and continuation tasks, improving the average rank from 2.79 to 1.80 and from 1.80 to 1.39, respectively.
Zitieren
APA
Wu, R., Dong, R., Pang, W., Pan, C., Zhang, R., Sun, H., Lu, H., & Fan, C. (2026). StoryCrafter: Instance-Aligned Multi-Character Storytelling with Diffusion Policy Learning and Test-Time Reinforcement Learning. ACM Transactions on Intelligent Systems and Technology. https://doi.org/10.1145/3856814
Vancouver
Wu R, Dong R, Pang W, Pan C, Zhang R, Sun H, et al. StoryCrafter: Instance-Aligned Multi-Character Storytelling with Diffusion Policy Learning and Test-Time Reinforcement Learning. ACM Transactions on Intelligent Systems and Technology. 2026. doi:10.1145/3856814
BibTeX
@article{Wu2026,
author = {Wu, Ruihan and Dong, Ruiqi and Pang, Wenjing and Pan, Chenjie and Zhang, Rui and Sun, Haishan and Lu, Heng-yang and Fan, Chenyou},
title = {StoryCrafter: Instance-Aligned Multi-Character Storytelling with Diffusion Policy Learning and Test-Time Reinforcement Learning},
journal = {ACM Transactions on Intelligent Systems and Technology},
year = {2026},
doi = {10.1145/3856814}
}
EUVIMED