Visual–Language Joint Modelling and Conditional Flow Matching for High‐Fidelity Cross‐Modal Animation Synthesis
IET Computer Vision · 2026-01 · 20 (1)
DOI: 10.1049/cvi2.70090 ↗Kurz gefasst
Fachartikel in der Zeitschrift IET Computer Vision, online erschienen am 05.11.2026. Verfasst von Siyu Song und 1 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Synthesising high‐fidelity animation from textual descriptions requires coordinated modelling of non‐rigid motion, semantic alignment and character appearance across video sequences. This study proposes a visual–language joint modelling framework that combines a multimodal large language model (MLLM)‐based semantic encoder, conditional flow matching (CFM) and a dual‐stream consistency module. The Q‐former adapter maps semantic features into the video latent space, whereas CFM parameterises continuous transport from noise towards encoded video data. Frequency‐domain filtering, reference attention, and an optical‐flow consistency loss are used to constrain appearance drift and inter‐frame artefacts. The framework is evaluated on AnimeShooter, with a curated WebVid‐10M subset incorporated during pretraining. The proposed model obtains an FVD of 82.16, a CLIPSIM of 0.348 and an adjacent‐frame ICS of 0.885.
Zitieren
APA
Song, S., & Li, C. (2026). Visual–Language Joint Modelling and Conditional Flow Matching for High‐Fidelity Cross‐Modal Animation Synthesis. IET Computer Vision, 20(1). https://doi.org/10.1049/cvi2.70090
Vancouver
Song S, Li C. Visual–Language Joint Modelling and Conditional Flow Matching for High‐Fidelity Cross‐Modal Animation Synthesis. IET Computer Vision. 2026;20(1). doi:10.1049/cvi2.70090
BibTeX
@article{Song2026,
author = {Song, Siyu and Li, Chuan},
title = {Visual–Language Joint Modelling and Conditional Flow Matching for High‐Fidelity Cross‐Modal Animation Synthesis},
journal = {IET Computer Vision},
year = {2026},
volume = {20},
number = {1},
doi = {10.1049/cvi2.70090}
}
EUVIMED