EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluation Methodologies in Software Protection Research

Bjorn De Sutter, Sebastian Schrittwieser, Bart Coppens, Patrick Kochberger

ACM Computing Surveys · 30.04.2025 · 57 (4) · 1-41

DOI: 10.1145/3702314 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 09.12.2024. Verfasst von Bjorn De Sutter und 3 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 202 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Austrian Science Fund, The Research Foundation - Flanders, Cybersecurity Research Program Flanders.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Man-at-the-end (MATE) attackers have full control over the system on which the attacked software runs, and try to break the confidentiality or integrity of assets embedded in the software. Both companies and malware authors want to prevent such attacks. This has driven an arms race between attackers and defenders, resulting in a plethora of different protection and analysis methods. However, it remains difficult to measure the strength of protections because MATE attackers can reach their goals in many different ways and a universally accepted evaluation methodology does not exist. This survey systematically reviews the evaluation methodologies of papers on obfuscation, a major class of protections against MATE attacks. For 571 papers, we collected 113 aspects of their evaluation methodologies, ranging from sample set types and sizes, over sample treatment, to performed measurements. We provide detailed insights into how the academic state of the art evaluates both the protections and analyses thereon. In summary, there is a clear need for better evaluation methodologies. We identify nine challenges for software protection evaluations, which represent threats to the validity, reproducibility, and interpretation of research results in the context of MATE attacks and formulate a number of concrete recommendations for improving the evaluations reported in future research papers.

Förderung

  • Austrian Science Fund
  • The Research Foundation - Flanders
  • Cybersecurity Research Program Flanders
  • Austrian Federal Ministry of Labour and Economy, the National Foundation for Research, Technology and Development, the Christian Doppler Research Association,…

Zitieren

APA

Sutter, B. D., Schrittwieser, S., Coppens, B., & Kochberger, P. (2025). Evaluation Methodologies in Software Protection Research. ACM Computing Surveys, 57(4), 1-41. https://doi.org/10.1145/3702314

Vancouver

Sutter BD, Schrittwieser S, Coppens B, Kochberger P. Evaluation Methodologies in Software Protection Research. ACM Computing Surveys. 2025;57(4):1-41. doi:10.1145/3702314

BibTeX
@article{Sutter2025,
  author = {Sutter, Bjorn De and Schrittwieser, Sebastian and Coppens, Bart and Kochberger, Patrick},
  title = {Evaluation Methodologies in Software Protection Research},
  journal = {ACM Computing Surveys},
  year = {2025},
  volume = {57},
  number = {4},
  pages = {1--41},
  doi = {10.1145/3702314}
}
Kostenlos starten →