EUVIMEDEUVIMED entdecken
Crossref

MulSMo: Multimodal Stylized Motion Generation by Bidirectional Control Flow

Zhe Li, Yisheng He, Lei Zhong, Weichao Shen, Qi Zuo, Lingteng Qiu, Shenhao Zhu, Zilong Dong, Laurence T Yang, Chang Xu, Weihao Yuan

IEEE Transactions on Image Processing · 2026 · 35 · 8719-8733

DOI: 10.1109/tip.2026.3719481 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2026. Verfasst von Zhe Li und 10 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 50 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Generating motion sequences conforming to a target style while adhering to the given content prompts requires accommodating both the content and style. In existing methods, the information usually only flows from style to content, which may cause conflict between the style and content, harming the integration. Differently, in this work we build a bidirectional control flow between the style and the content, also adjusting the style towards the content, in which case the style-content collision is alleviated and the dynamics of the style is better preserved in the integration. Moreover, we extend the stylized motion generation from one modality, i.e. the style motion, to multiple modalities including texts and images through contrastive learning, leading to flexible style control on the motion generation. To further boost the performance, we advance the motion diffusion to motion-aligned temporal latent diffusion by developing a novel motion VAE. Extensive experiments demonstrate that our method significantly outperforms previous methods across different datasets, while also enabling multimodal signals control. The code of our method will be made publicly available.

Zitieren

APA

Li, Z., He, Y., Zhong, L., Shen, W., Zuo, Q., Qiu, L., Zhu, S., Dong, Z., Yang, L. T., Xu, C., & Yuan, W. (2026). MulSMo: Multimodal Stylized Motion Generation by Bidirectional Control Flow. IEEE Transactions on Image Processing, 35, 8719-8733. https://doi.org/10.1109/tip.2026.3719481

Vancouver

Li Z, He Y, Zhong L, Shen W, Zuo Q, Qiu L, et al. MulSMo: Multimodal Stylized Motion Generation by Bidirectional Control Flow. IEEE Transactions on Image Processing. 2026;35:8719-8733. doi:10.1109/tip.2026.3719481

BibTeX
@article{Li2026,
  author = {Li, Zhe and He, Yisheng and Zhong, Lei and Shen, Weichao and Zuo, Qi and Qiu, Lingteng and Zhu, Shenhao and Dong, Zilong and Yang, Laurence T and Xu, Chang and Yuan, Weihao},
  title = {MulSMo: Multimodal Stylized Motion Generation by Bidirectional Control Flow},
  journal = {IEEE Transactions on Image Processing},
  year = {2026},
  volume = {35},
  pages = {8719--8733},
  doi = {10.1109/tip.2026.3719481}
}
Kostenlos starten →