EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation

Shijie Ma, Huayi Xu, Mengjian Li, Haiyang Li, Xuemei Yang, Yongrui Li, Yujiao Wu, Yaxiong Wang

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.08.2026 · 22 (8) · 1-17

DOI: 10.1145/3820056 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 22.07.2026. Verfasst von Shijie Ma und 7 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This article targets to enhance the diffusion-based text-to-video generation by improving the two input prompts, including the noise and the text. Accommodated with this goal, we propose POS, a P rompt O ptimization S uite to boost text-to-video models. POS is motivated by two observations: (1) Video generation shows instability in terms of noise . Given the same text, different noises lead to videos that differ significantly in terms of both frame quality and temporal consistency. This observation implies that there exists an optimal noise matched to each textual input; To capture the potential noise, we propose an optimal noise approximator to approach the potential optimal noise. Particularly, the optimal noise approximator initially searches a video that closely relates to the text prompt and then inverts it into the noise space to serve as an improved noise prompt for the textual input. (2) Improving the text prompt via LLMs often causes semantic deviation . Many existing text-to-vision works have utilized LLMs to improve the text prompts for generation enhancement. However, existing methods often neglect the semantic alignment between the original text and the rewritten one. In response to this issue, we design a semantic-preserving rewriter to impose constraints in both rewriting and denoising phrases to preserve semantic consistency. Extensive experiments on popular benchmarks show that our POS can improve the text-to-video models with a clear margin.

Förderung

  • National Key Research and Development Program of China

Zitieren

APA

Ma, S., Xu, H., Li, M., Li, H., Yang, X., Li, Y., Wu, Y., & Wang, Y. (2026). POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(8), 1-17. https://doi.org/10.1145/3820056

Vancouver

Ma S, Xu H, Li M, Li H, Yang X, Li Y, et al. POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(8):1-17. doi:10.1145/3820056

BibTeX
@article{Ma2026,
  author = {Ma, Shijie and Xu, Huayi and Li, Mengjian and Li, Haiyang and Yang, Xuemei and Li, Yongrui and Wu, Yujiao and Wang, Yaxiong},
  title = {POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2026},
  volume = {22},
  number = {8},
  pages = {1--17},
  doi = {10.1145/3820056}
}
Kostenlos starten →