How Composite Metamorphic Relations Enhance Test Effectiveness of DNN Testing: An Empirical Study
IEEE Transactions on Software Engineering · 2026-04 · 52 (4) · 1617-1636
DOI: 10.1109/tse.2026.3675285 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Software Engineering, 2026-04. Verfasst von Huayao Wu und 5 weiteren Autor:innen. Die Arbeit nennt 89 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Jiangsu Provincial Key Research and Development Program, Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China.
Abstract (Originalsprache)
Metamorphic Testing (MT) is a powerful technique to alleviate the test oracle problem of DNN testing. At its core is a set of Metamorphic Relations (MRs), which indicate necessary properties that multiple test inputs and their outputs should satisfy. Recently, researchers have proposed to compose multiple individual MRs to create new Composite Metamorphic Relations (CMRs), in the hope of enhancing the cost-effectiveness of MT. However, the potential benefit of such compositions is yet to be systematically explored for DNN based systems. In this paper, we present the first empirical study that investigates the relative test effectiveness of CMRs over individual MRs in DNN testing. Our experiment is performed under five popular image recognition models and seven representative component MRs, based on which a total of 3,612 CMRs and more than 200 million pairs of source and follow-up test images are generated and exercised. The experimental results highlight the advantages of employing CMRs in DNN testing, as they generally exhibit superior failure and fault revelation effectiveness compared to any individual MR. Moreover, to further illuminate the conditions that allow CMRs to maximise their performance in DNN testing, we propose to analyse the geometric relationships of MRs in the latent embedding space of DNNs, thereby quantifying the extent to which different MRs complement each other. Accordingly, by opting for highly complementary MRs in composition, we can create CMRs that are most likely to enhance test effectiveness.
Förderung
- National Natural Science Foundation of China
- Jiangsu Provincial Key Research and Development Program
- Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China
Zitieren
APA
Wu, H., Wang, P., Hu, S., Niu, X., Nie, C., & Chen, T. Y. (2026). How Composite Metamorphic Relations Enhance Test Effectiveness of DNN Testing: An Empirical Study. IEEE Transactions on Software Engineering, 52(4), 1617-1636. https://doi.org/10.1109/tse.2026.3675285
Vancouver
Wu H, Wang P, Hu S, Niu X, Nie C, Chen TY. How Composite Metamorphic Relations Enhance Test Effectiveness of DNN Testing: An Empirical Study. IEEE Transactions on Software Engineering. 2026;52(4):1617-1636. doi:10.1109/tse.2026.3675285
BibTeX
@article{Wu2026,
author = {Wu, Huayao and Wang, Peng and Hu, Shengyou and Niu, Xintao and Nie, Changhai and Chen, Tsong Yueh},
title = {How Composite Metamorphic Relations Enhance Test Effectiveness of DNN Testing: An Empirical Study},
journal = {IEEE Transactions on Software Engineering},
year = {2026},
volume = {52},
number = {4},
pages = {1617--1636},
doi = {10.1109/tse.2026.3675285}
}
EUVIMED