Evaluating the Generalization Ability of Spatiotemporal Model in Urban Scenario
IEEE Transactions on Mobile Computing · 2025-12 · 24 (12) · 13535-13548
DOI: 10.1109/tmc.2025.3590606 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2025-12. Verfasst von Hongjun Wang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 82 Literaturangaben. Gefördert von: Jilin Provincial International Cooperation Key Laboratory for Super Smart City and Jilin Provincial Key Laboratory of Intelligent Policing.
Abstract (Originalsprache)
Spatiotemporal neural networks have shown great promise in urban scenarios by effectively capturing temporal and spatial correlations. However, urban environments are constantly evolving, and current model evaluations are often limited to traffic scenarios and use data mainly collected only a few weeks after training period to evaluate model performance. The generalization ability of these models remains largely unexplored. To address this, we propose a Spatiotemporal Out-of-Distribution (ST-OOD) benchmark, which comprises six urban scenario: bike-sharing, 311 services, pedestrian counts, traffic speed, traffic flow, ride-hailing demand, and bike-sharing, each with in-distribution (same year) and out-of-distribution (next years) settings. We extensively evaluate state-of-the-art spatiotemporal models and find that their performance degrades significantly in out-of-distribution settings, with most models performing even worse than a simple Multi-Layer Perceptron (MLP). Our findings suggest that current leading methods tend to over-rely on parameters to overfit training data, which may lead to good performance on in-distribution data but often results in poor generalization. We also investigated whether dropout could mitigate the negative effects of overfitting. Our results showed that a slight dropout rate could significantly improve generalization performance on most datasets, with minimal impact on in-distribution performance. However, balancing in-distribution and out-of-distribution performance remains a challenging problem. We hope that the proposed benchmark will encourage further research on this critical issue.
Förderung
- Jilin Provincial International Cooperation Key Laboratory for Super Smart City and Jilin Provincial Key Laboratory of Intelligent Policing
Zitieren
APA
Wang, H., Chen, J., Pan, T., Dong, Z., Jiang, R., & Song, X. (2025). Evaluating the Generalization Ability of Spatiotemporal Model in Urban Scenario. IEEE Transactions on Mobile Computing, 24(12), 13535-13548. https://doi.org/10.1109/tmc.2025.3590606
Vancouver
Wang H, Chen J, Pan T, Dong Z, Jiang R, Song X. Evaluating the Generalization Ability of Spatiotemporal Model in Urban Scenario. IEEE Transactions on Mobile Computing. 2025;24(12):13535-13548. doi:10.1109/tmc.2025.3590606
BibTeX
@article{Wang2025,
author = {Wang, Hongjun and Chen, Jiyuan and Pan, Tong and Dong, Zheng and Jiang, Renhe and Song, Xuan},
title = {Evaluating the Generalization Ability of Spatiotemporal Model in Urban Scenario},
journal = {IEEE Transactions on Mobile Computing},
year = {2025},
volume = {24},
number = {12},
pages = {13535--13548},
doi = {10.1109/tmc.2025.3590606}
}
EUVIMED