MarMot: Metamorphic Runtime Monitoring of Autonomous Driving Systems
ACM Transactions on Software Engineering and Methodology · 31.01.2025 · 34 (1) · 1-35
DOI: 10.1145/3678171 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 27.12.2024. Verfasst von Jon Ayerdi und 5 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: Basque Government through their Elkartek, Department of Education, Universities and Research of the Basque Country.
Abstract (Originalsprache)
Autonomous driving systems (ADSs) are complex cyber-physical systems (CPSs) that must ensure safety even in uncertain conditions. Modern ADSs often employ deep neural networks (DNNs), which may not produce correct results in every possible driving scenario. Thus, an approach to estimate the confidence of an ADS at runtime is necessary to prevent potentially dangerous situations. In this article we propose MarMot , an online monitoring approach for ADSs based on metamorphic relations (MRs), which are properties of a system that hold among multiple inputs and the corresponding outputs. Using domain-specific MRs, MarMot estimates the uncertainty of the ADS at runtime, allowing the identification of anomalous situations that are likely to cause a faulty behavior of the ADS, such as driving off the road. We perform an empirical assessment of MarMot with five different MRs, using two different subject ADSs, including a small-scale physical ADS and a simulated ADS. Our evaluation encompasses the identification of both external anomalies, e.g., fog, as well as internal anomalies, e.g., faulty DNNs due to mislabeled training data. Our results show that MarMot can identify up to 65% of the external anomalies and 100% of the internal anomalies in the physical ADS, and up to 54% of the external anomalies and 88% of the internal anomalies in the simulated ADS. With these results, MarMot outperforms or is comparable to other state-of-the-art approaches, including SelfOracle, Ensemble, and MC Dropout-based ADS monitors.
Förderung
- Basque Government through their Elkartek
- Department of Education, Universities and Research of the Basque Country
Zitieren
APA
Ayerdi, J., Iriarte, A., Valle, P., Roman, I., Illarramendi, M., & Arrieta, A. (2025). MarMot: Metamorphic Runtime Monitoring of Autonomous Driving Systems. ACM Transactions on Software Engineering and Methodology, 34(1), 1-35. https://doi.org/10.1145/3678171
Vancouver
Ayerdi J, Iriarte A, Valle P, Roman I, Illarramendi M, Arrieta A. MarMot: Metamorphic Runtime Monitoring of Autonomous Driving Systems. ACM Transactions on Software Engineering and Methodology. 2025;34(1):1-35. doi:10.1145/3678171
BibTeX
@article{Ayerdi2025,
author = {Ayerdi, Jon and Iriarte, Asier and Valle, Pablo and Roman, Ibai and Illarramendi, Miren and Arrieta, Aitor},
title = {MarMot: Metamorphic Runtime Monitoring of Autonomous Driving Systems},
journal = {ACM Transactions on Software Engineering and Methodology},
year = {2025},
volume = {34},
number = {1},
pages = {1--35},
doi = {10.1145/3678171}
}
EUVIMED