EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

UniTriM: Unified Text–Image–Video Retrieval via Multi-Granular Alignment and Feature Disentanglement

Yangchen Wang, Yan Hua, Yingyun Yang, Wenhui Zhang

Electronics · 30.03.2026 · 15 (7) · 1424

DOI: 10.3390/electronics15071424 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 30.03.2026. Verfasst von Yangchen Wang und 3 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the proliferation of multimodal content on social media, creators increasingly require tools that can retrieve both images and videos relevant to a single textual query. However, existing cross-modal retrieval methods are typically confined to binary (text–image or text–video) settings and struggle with fine-grained semantic alignment and spatiotemporal information imbalance. To address this issue, we propose UniTriM, a unified framework for text–image–video joint retrieval. First, UniTriM supports concurrent retrieval of semantically relevant images and videos given one textual input. To overcome the scarcity of text–image–video triplet data, we introduce a self-attention-based keyframe selection strategy that converts existing text–video datasets into triplet format. Second, we design a multi-granularity similarity alignment module that captures hierarchical semantics by modeling patch–frame–video and word–triple–sentence structures and jointly optimizes intra- and cross-granularity alignments to enhance fine-grained cross-modal correspondence. Third, to alleviate the inherent spatiotemporal information imbalance between static images and video-aligned text descriptions, we introduce a feature disentanglement module that disentangles spatial-related features from text and aligns them explicitly with image representations. Experiments conducted on three benchmark datasets MSR-VTT, MSVD, and DiDeMo demonstrate that UniTriM achieves state-of-the-art performance on joint retrieval tasks.

Zitieren

APA

Wang, Y., Hua, Y., Yang, Y., & Zhang, W. (2026). UniTriM: Unified Text–Image–Video Retrieval via Multi-Granular Alignment and Feature Disentanglement. Electronics, 15(7), 1424. https://doi.org/10.3390/electronics15071424

Vancouver

Wang Y, Hua Y, Yang Y, Zhang W. UniTriM: Unified Text–Image–Video Retrieval via Multi-Granular Alignment and Feature Disentanglement. Electronics. 2026;15(7):1424. doi:10.3390/electronics15071424

BibTeX
@article{Wang2026,
  author = {Wang, Yangchen and Hua, Yan and Yang, Yingyun and Zhang, Wenhui},
  title = {UniTriM: Unified Text–Image–Video Retrieval via Multi-Granular Alignment and Feature Disentanglement},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {7},
  pages = {1424},
  doi = {10.3390/electronics15071424}
}
Kostenlos starten →