EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Multidimensional Framework for Analyzing Image–Text Consistency in Social Media

Hongqi Xia, Zhijie Zhao, Binbin Zhao, Hong Lan, Han Wu, Xujing Jing, Yanrong Zhang

Applied Sciences · 21.04.2026 · 16 (8) · 4044

DOI: 10.3390/app16084044 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 21.04.2026. Verfasst von Hongqi Xia und 6 weiteren Autor:innen. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Heilongjiang Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As image–text posts have become a dominant form of social media communication, understanding how the two modalities jointly convey meaning remains a key challenge in multimodal analysis. This study aims to examine whether image–text consistency is inherently multidimensional rather than reducible to a single similarity metric. Existing studies often reduce consistency to a single relevance score, which cannot capture semantic, emotional, and functional interactions. We construct a dataset of 28,650 multimodal posts and model image–text relationships along three dimensions: semantic consistency (CSC), emotional consistency (CEC), and informational matching consistency (IMC). Semantic and emotional alignment are measured using cross-modal representation and similarity computation, while IMC is defined through rule-based classification of informational roles. Results show that emotional consistency (CEC = 0.621) is higher than semantic consistency (CSC = 0.549, p

Förderung

  • Natural Science Foundation of Heilongjiang Province

Zitieren

APA

Xia, H., Zhao, Z., Zhao, B., Lan, H., Wu, H., Jing, X., & Zhang, Y. (2026). A Multidimensional Framework for Analyzing Image–Text Consistency in Social Media. Applied Sciences, 16(8), 4044. https://doi.org/10.3390/app16084044

Vancouver

Xia H, Zhao Z, Zhao B, Lan H, Wu H, Jing X, et al. A Multidimensional Framework for Analyzing Image–Text Consistency in Social Media. Applied Sciences. 2026;16(8):4044. doi:10.3390/app16084044

BibTeX
@article{Xia2026,
  author = {Xia, Hongqi and Zhao, Zhijie and Zhao, Binbin and Lan, Hong and Wu, Han and Jing, Xujing and Zhang, Yanrong},
  title = {A Multidimensional Framework for Analyzing Image–Text Consistency in Social Media},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {8},
  pages = {4044},
  doi = {10.3390/app16084044}
}
Kostenlos starten →