EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Visual–Language Joint Modelling and Conditional Flow Matching for High‐Fidelity Cross‐Modal Animation Synthesis

Siyu Song, Chuan Li

IET Computer Vision · 2026-01 · 20 (1)

DOI: 10.1049/cvi2.70090 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IET Computer Vision, online erschienen am 05.11.2026. Verfasst von Siyu Song und 1 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Synthesising high‐fidelity animation from textual descriptions requires coordinated modelling of non‐rigid motion, semantic alignment and character appearance across video sequences. This study proposes a visual–language joint modelling framework that combines a multimodal large language model (MLLM)‐based semantic encoder, conditional flow matching (CFM) and a dual‐stream consistency module. The Q‐former adapter maps semantic features into the video latent space, whereas CFM parameterises continuous transport from noise towards encoded video data. Frequency‐domain filtering, reference attention, and an optical‐flow consistency loss are used to constrain appearance drift and inter‐frame artefacts. The framework is evaluated on AnimeShooter, with a curated WebVid‐10M subset incorporated during pretraining. The proposed model obtains an FVD of 82.16, a CLIPSIM of 0.348 and an adjacent‐frame ICS of 0.885.

Zitieren

APA

Song, S., & Li, C. (2026). Visual–Language Joint Modelling and Conditional Flow Matching for High‐Fidelity Cross‐Modal Animation Synthesis. IET Computer Vision, 20(1). https://doi.org/10.1049/cvi2.70090

Vancouver

Song S, Li C. Visual–Language Joint Modelling and Conditional Flow Matching for High‐Fidelity Cross‐Modal Animation Synthesis. IET Computer Vision. 2026;20(1). doi:10.1049/cvi2.70090

BibTeX
@article{Song2026,
  author = {Song, Siyu and Li, Chuan},
  title = {Visual–Language Joint Modelling and Conditional Flow Matching for High‐Fidelity Cross‐Modal Animation Synthesis},
  journal = {IET Computer Vision},
  year = {2026},
  volume = {20},
  number = {1},
  doi = {10.1049/cvi2.70090}
}
Kostenlos starten →