POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation
ACM Transactions on Multimedia Computing, Communications, and Applications · 31.08.2026 · 22 (8) · 1-17
DOI: 10.1145/3820056 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 22.07.2026. Verfasst von Shijie Ma und 7 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China.
Abstract (Originalsprache)
This article targets to enhance the diffusion-based text-to-video generation by improving the two input prompts, including the noise and the text. Accommodated with this goal, we propose POS, a P rompt O ptimization S uite to boost text-to-video models. POS is motivated by two observations: (1) Video generation shows instability in terms of noise . Given the same text, different noises lead to videos that differ significantly in terms of both frame quality and temporal consistency. This observation implies that there exists an optimal noise matched to each textual input; To capture the potential noise, we propose an optimal noise approximator to approach the potential optimal noise. Particularly, the optimal noise approximator initially searches a video that closely relates to the text prompt and then inverts it into the noise space to serve as an improved noise prompt for the textual input. (2) Improving the text prompt via LLMs often causes semantic deviation . Many existing text-to-vision works have utilized LLMs to improve the text prompts for generation enhancement. However, existing methods often neglect the semantic alignment between the original text and the rewritten one. In response to this issue, we design a semantic-preserving rewriter to impose constraints in both rewriting and denoising phrases to preserve semantic consistency. Extensive experiments on popular benchmarks show that our POS can improve the text-to-video models with a clear margin.
Förderung
- National Key Research and Development Program of China
Zitieren
APA
Ma, S., Xu, H., Li, M., Li, H., Yang, X., Li, Y., Wu, Y., & Wang, Y. (2026). POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(8), 1-17. https://doi.org/10.1145/3820056
Vancouver
Ma S, Xu H, Li M, Li H, Yang X, Li Y, et al. POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(8):1-17. doi:10.1145/3820056
BibTeX
@article{Ma2026,
author = {Ma, Shijie and Xu, Huayi and Li, Mengjian and Li, Haiyang and Yang, Xuemei and Li, Yongrui and Wu, Yujiao and Wang, Yaxiong},
title = {POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2026},
volume = {22},
number = {8},
pages = {1--17},
doi = {10.1145/3820056}
}
EUVIMED