EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Traceable Metamorphic Test Cases for Robust Safety‐Critical Systems: A Deep Learning LiDAR Object Detector Example

Simon Speth, Tobias Springer, Jannik Fischbach, Alexander Pretschner

Software Testing, Verification and Reliability · 2026-05 · 36 (3-4)

DOI: 10.1002/stvr.70020 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Software Testing, Verification and Reliability, online erschienen am 23.04.2026. Verfasst von Simon Speth und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Assessing the robustness of safety‐critical deep learning (DL) systems is of utmost importance, as these systems can cause harm when deployed in the real world. Metamorphic testing (MT) is one commonly used method to evaluate the robustness of DL systems, as it does not require expensive labelled ground truth data. This paper tackles two challenges: (1) One challenge in regulated domains such as the automotive industry is to provide a traceable argumentation of why a certain metamorphic relation (MR) was chosen. We adopt the idea of defect‐based testing to MT and argue that an MR is traceable if it can be linked to a defect hypothesis. We demonstrate how to assess the robustness of safety‐critical DL systems using the example of LiDAR object detectors. To this end, we create three new MRs for the LiDAR domain and identify five MR that can be reused by adapting them from related domains. Our experiments on the nuScenes dataset with three different object detectors produce 3.9 million test verdicts, of which 0.7 million are test failures. This shows that our defect‐based MR effectively uncover failures. (2) A second challenge resulting from executing numerous metamorphic test cases is that MT can lead to the generation of an impractically high number of failures. We show how to prioritize the most critical failures, such as failures that occur close to the ego vehicle. By prioritizing, we reduced the observed 685,000 failures to 5397 safety‐critical failures corresponding to a 127‐fold reduction.

Zitieren

APA

Speth, S., Springer, T., Fischbach, J., & Pretschner, A. (2026). Traceable Metamorphic Test Cases for Robust Safety‐Critical Systems: A Deep Learning LiDAR Object Detector Example. Software Testing, Verification and Reliability, 36(3-4). https://doi.org/10.1002/stvr.70020

Vancouver

Speth S, Springer T, Fischbach J, Pretschner A. Traceable Metamorphic Test Cases for Robust Safety‐Critical Systems: A Deep Learning LiDAR Object Detector Example. Software Testing, Verification and Reliability. 2026;36(3-4). doi:10.1002/stvr.70020

BibTeX
@article{Speth2026,
  author = {Speth, Simon and Springer, Tobias and Fischbach, Jannik and Pretschner, Alexander},
  title = {Traceable Metamorphic Test Cases for Robust Safety‐Critical Systems: A Deep Learning LiDAR Object Detector Example},
  journal = {Software Testing, Verification and Reliability},
  year = {2026},
  volume = {36},
  number = {3-4},
  doi = {10.1002/stvr.70020}
}
Kostenlos starten →