UniTriM: Unified Text–Image–Video Retrieval via Multi-Granular Alignment and Feature Disentanglement
Electronics · 30.03.2026 · 15 (7) · 1424
DOI: 10.3390/electronics15071424 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 30.03.2026. Verfasst von Yangchen Wang und 3 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
With the proliferation of multimodal content on social media, creators increasingly require tools that can retrieve both images and videos relevant to a single textual query. However, existing cross-modal retrieval methods are typically confined to binary (text–image or text–video) settings and struggle with fine-grained semantic alignment and spatiotemporal information imbalance. To address this issue, we propose UniTriM, a unified framework for text–image–video joint retrieval. First, UniTriM supports concurrent retrieval of semantically relevant images and videos given one textual input. To overcome the scarcity of text–image–video triplet data, we introduce a self-attention-based keyframe selection strategy that converts existing text–video datasets into triplet format. Second, we design a multi-granularity similarity alignment module that captures hierarchical semantics by modeling patch–frame–video and word–triple–sentence structures and jointly optimizes intra- and cross-granularity alignments to enhance fine-grained cross-modal correspondence. Third, to alleviate the inherent spatiotemporal information imbalance between static images and video-aligned text descriptions, we introduce a feature disentanglement module that disentangles spatial-related features from text and aligns them explicitly with image representations. Experiments conducted on three benchmark datasets MSR-VTT, MSVD, and DiDeMo demonstrate that UniTriM achieves state-of-the-art performance on joint retrieval tasks.
Zitieren
APA
Wang, Y., Hua, Y., Yang, Y., & Zhang, W. (2026). UniTriM: Unified Text–Image–Video Retrieval via Multi-Granular Alignment and Feature Disentanglement. Electronics, 15(7), 1424. https://doi.org/10.3390/electronics15071424
Vancouver
Wang Y, Hua Y, Yang Y, Zhang W. UniTriM: Unified Text–Image–Video Retrieval via Multi-Granular Alignment and Feature Disentanglement. Electronics. 2026;15(7):1424. doi:10.3390/electronics15071424
BibTeX
@article{Wang2026,
author = {Wang, Yangchen and Hua, Yan and Yang, Yingyun and Zhang, Wenhui},
title = {UniTriM: Unified Text–Image–Video Retrieval via Multi-Granular Alignment and Feature Disentanglement},
journal = {Electronics},
year = {2026},
volume = {15},
number = {7},
pages = {1424},
doi = {10.3390/electronics15071424}
}
EUVIMED