A Multidimensional Framework for Analyzing Image–Text Consistency in Social Media
Applied Sciences · 21.04.2026 · 16 (8) · 4044
DOI: 10.3390/app16084044 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 21.04.2026. Verfasst von Hongqi Xia und 6 weiteren Autor:innen. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Heilongjiang Province.
Abstract (Originalsprache)
As image–text posts have become a dominant form of social media communication, understanding how the two modalities jointly convey meaning remains a key challenge in multimodal analysis. This study aims to examine whether image–text consistency is inherently multidimensional rather than reducible to a single similarity metric. Existing studies often reduce consistency to a single relevance score, which cannot capture semantic, emotional, and functional interactions. We construct a dataset of 28,650 multimodal posts and model image–text relationships along three dimensions: semantic consistency (CSC), emotional consistency (CEC), and informational matching consistency (IMC). Semantic and emotional alignment are measured using cross-modal representation and similarity computation, while IMC is defined through rule-based classification of informational roles. Results show that emotional consistency (CEC = 0.621) is higher than semantic consistency (CSC = 0.549, p
Förderung
- Natural Science Foundation of Heilongjiang Province
Zitieren
APA
Xia, H., Zhao, Z., Zhao, B., Lan, H., Wu, H., Jing, X., & Zhang, Y. (2026). A Multidimensional Framework for Analyzing Image–Text Consistency in Social Media. Applied Sciences, 16(8), 4044. https://doi.org/10.3390/app16084044
Vancouver
Xia H, Zhao Z, Zhao B, Lan H, Wu H, Jing X, et al. A Multidimensional Framework for Analyzing Image–Text Consistency in Social Media. Applied Sciences. 2026;16(8):4044. doi:10.3390/app16084044
BibTeX
@article{Xia2026,
author = {Xia, Hongqi and Zhao, Zhijie and Zhao, Binbin and Lan, Hong and Wu, Han and Jing, Xujing and Zhang, Yanrong},
title = {A Multidimensional Framework for Analyzing Image–Text Consistency in Social Media},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {8},
pages = {4044},
doi = {10.3390/app16084044}
}
EUVIMED