From words to visuals: a transformer-based multi-modal framework for emotion-driven tourism analytics
Information Technology & Tourism · 2025-12 · 27 (4) · 939-979
DOI: 10.1007/s40558-025-00334-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift Information Technology & Tourism, online erschienen am 22.07.2025. Verfasst von Víctor Calderón-Fajardo und 2 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 82 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Universidad de Málaga.
Abstract (Originalsprache)
Traditional tourism analytics have primarily relied on isolated sentiment analysis and image processing techniques, often failing to capture the subtle interaction between textual expressions and visual aesthetics inherent in tourist experiences. This study addresses these limitations by proposing a novel multi-modal framework that transforms textual reviews into AI-generated images using standardized prompts, thereby converting affective signals into explicit visual features. Leveraging state-of-the-art models—such as Distilled Bidirectional Encoder Representations from Transformers (DistilBERT) for fine-grained emotion recognition and Contrastive Language–Image Pre‑training (CLIP) for semantic extraction of visual attributes—our approach maps complex sentiments onto interpretable visual characteristics, integrating explainable features to uncover the underlying structure in tourist perceptions. This approach enhances classification performance and provides a transparent mechanism for understanding how distinct emotional states correspond to specific visual cues. Experimental evaluations on a dataset encompassing four diverse tourist destinations—Berlin, Dublin, Cairo, and Málaga—demonstrate high classification accuracy and robust correlations between text-derived emotions and image-based features, close to more powerful embedding methods. Significant correlations were observed between emotions and visual features, e.g., brightness and contentment, as well as between entropy and shame, indicating that our method efficiently captures the affective resonance between visual and textual modalities. Our findings underscore the transformative potential of converting textual sentiment into visual representations to facilitate more accurate, interpretable, and actionable analytics in the tourism sector. This framework suggests promising avenues for dynamic destination characterization, informed marketing strategies, and enhanced urban planning initiatives, laying the foundation for future advancements in multi-modal tourism analytics.
Förderung
- Universidad de Málaga
Zitieren
APA
Calderón-Fajardo, V., Rodríguez-Rodríguez, I., & Puig-Cabrera, M. (2025). From words to visuals: a transformer-based multi-modal framework for emotion-driven tourism analytics. Information Technology & Tourism, 27(4), 939-979. https://doi.org/10.1007/s40558-025-00334-2
Vancouver
Calderón-Fajardo V, Rodríguez-Rodríguez I, Puig-Cabrera M. From words to visuals: a transformer-based multi-modal framework for emotion-driven tourism analytics. Information Technology & Tourism. 2025;27(4):939-979. doi:10.1007/s40558-025-00334-2
BibTeX
@article{CaldernFajardo2025,
author = {Calderón-Fajardo, Víctor and Rodríguez-Rodríguez, Ignacio and Puig-Cabrera, Miguel},
title = {From words to visuals: a transformer-based multi-modal framework for emotion-driven tourism analytics},
journal = {Information Technology & Tourism},
year = {2025},
volume = {27},
number = {4},
pages = {939--979},
doi = {10.1007/s40558-025-00334-2}
}
EUVIMED