Multi-Modal Artificial Intelligence for Smart Cities: Experimental Integration of Textual and Sensor Data
Future Internet · 05.03.2026 · 18 (3) · 136
DOI: 10.3390/fi18030136 ↗Kurz gefasst
Fachartikel in der Zeitschrift Future Internet, online erschienen am 05.03.2026. Verfasst von Nouf Alkhater. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Smart city decision-making increasingly relies on heterogeneous urban data sources. Dense traffic sensor streams provide continuous quantitative measurements, while citizen-generated textual reports offer event-driven contextual information. However, integrating these modalities remains challenging due to temporal misalignment, textual sparsity, and semantic noise. This paper investigates multi-modal learning for traffic congestion severity prediction through an experimental integration of open traffic sensor data (METR-LA: Los Angeles, USA) and citizen-generated textual reports (NYC 311: New York City, USA). Congestion severity is formulated as a four-class classification task derived from traffic speed measurements. We propose an end-to-end framework that combines: (i) sensor time-series encoding using a GRU-based temporal encoder, (ii) textual representation learning using a BERT-based encoder, (iii) a symmetric time-window alignment strategy (±Δ) to associate irregular reports with sensor time steps, and (iv) multiple fusion architectures, including early fusion, late fusion, and a cross-attention module for cross-modal interaction modeling. Experiments on publicly available datasets show that multi-modal early fusion achieves the best overall performance (Accuracy = 0.8283, Macro-F1 = 0.8231) compared to uni-modal baselines. In the studied cross-city setting with sparse and weakly aligned textual signals, the proposed cross-attention fusion does not outperform the strong sensor-only baseline, suggesting that the sensor modality dominates when cross-modal signal strength is limited. These results highlight both the potential and the practical constraints of multi-modal fusion in heterogeneous smart-city environments, emphasizing the importance of alignment design, modality relevance, and transparent experimental validation.
Zitieren
APA
Alkhater, N. (2026). Multi-Modal Artificial Intelligence for Smart Cities: Experimental Integration of Textual and Sensor Data. Future Internet, 18(3), 136. https://doi.org/10.3390/fi18030136
Vancouver
Alkhater N. Multi-Modal Artificial Intelligence for Smart Cities: Experimental Integration of Textual and Sensor Data. Future Internet. 2026;18(3):136. doi:10.3390/fi18030136
BibTeX
@article{Alkhater2026,
author = {Alkhater, Nouf},
title = {Multi-Modal Artificial Intelligence for Smart Cities: Experimental Integration of Textual and Sensor Data},
journal = {Future Internet},
year = {2026},
volume = {18},
number = {3},
pages = {136},
doi = {10.3390/fi18030136}
}
EUVIMED