Towards Defect-based Testing for Safety-critical ML Components
Safety-Critical Systems eJournal · 2024
DOI: 10.65391/r3146 ↗Kurz gefasst
Fachartikel in der Zeitschrift Safety-Critical Systems eJournal, 2024. Verfasst von Amit Sahu und 1 weiteren Autor:innen.
Abstract (Originalsprache)
The input space of Machine Learning (ML) components used in safety-critical applications is complex. Testing such components on exponentially large datasets that cover all potential real-world situations to meaningfully measure performance metrics, such as the false negative rate, is infeasible due to practical restrictions. Consequently, we must limit the test data while adequately covering critical input data points. Inspired by defect-based software testing, a method for specifying adequate test cases for software components, we propose a process for collecting adequate test data for ML components. Concretely, we systematically employ different existing ML data quality metrics, and methods for enhancing the test data, to uncover critical scenarios where the ML component may be less performant. We exemplify the usage of our process in two case studies, each involving an ML component implementing different functionalities, i.e. stop sign recognition, and railway track segmentation.
Zitieren
APA
Sahu, A., & Cârlan, C. (2024). Towards Defect-based Testing for Safety-critical ML Components. Safety-Critical Systems eJournal. https://doi.org/10.65391/r3146
Vancouver
Sahu A, Cârlan C. Towards Defect-based Testing for Safety-critical ML Components. Safety-Critical Systems eJournal. 2024. doi:10.65391/r3146
BibTeX
@article{Sahu2024,
author = {Sahu, Amit and Cârlan, Carmen},
title = {Towards Defect-based Testing for Safety-critical ML Components},
journal = {Safety-Critical Systems eJournal},
year = {2024},
doi = {10.65391/r3146}
}
EUVIMED