EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Beyond Predictive Performance: SHAP-Based Explanations and Alignment with Literature-Based Indicators in Code Smell Detection

Marcela Mosquera, Rodolfo Bojorque

Information · 20.11.2026 · 17 (11) · 1184

DOI: 10.3390/info17111184 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information, online erschienen am 20.11.2026. Verfasst von Marcela Mosquera und 1 weiteren Autor:innen. Die Arbeit nennt 61 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Universidad Politécnica Salesiana.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Machine learning-based code smell detection relies on structural metrics, but predictive performance alone does not establish whether the information used by classifiers corresponds to the characteristics reported in the literature. This study examines the SHapley Additive exPlanations (SHAP)-based attribution behavior of Support Vector Machine (SVM) models by integrating SHAP-based feature importance, point-biserial associations, and 40 literature-based code smell operationalizations. The analysis reconstructs 120 SVM models across three open-source Java systems and four code smells, using ten folds per system–smell combination. Feature importance, persistence across folds, and consistency across software systems are examined alongside SHAP–correlation concordance, indicator coverage, and directional compatibility. The results show that influential metrics vary across code smells and software systems, while high average importance does not necessarily imply persistence across folds. For example, NOD achieved the highest mean normalized importance for Azureus–God Class (0.211) but appeared in the fold-level Top-5 in only 30% of the folds. Agreement between SHAP-based rankings and statistical associations is partial, with a median Top-5 overlap of 0.30. Indicator coverage and directional compatibility provide distinct forms of evidence; for God Class, mean Top-5 coverage was 0.05 whereas mean directional compatibility was 0.92, and some literature-based operationalizations cannot be evaluated using the available metrics. Additional sensitivity and robustness analyses showed that the main findings were generally stable across alternative Top-k cutoffs, association partitions, metric-mapping scenarios, and SHAP configurations; repeated SHAP realizations showed particularly high agreement for background sizes of 50 or more instances. These findings indicate that explanatory relevance depends on the experimental context and that model contributions, statistical associations, and literature-based indicators provide complementary evidence for interpreting code smell predictions. The results are relevant to researchers developing explainable code smell detectors and to tool developers and practitioners assessing whether predictive models rely on structural evidence consistent with established smell definitions.

Förderung

  • Universidad Politécnica Salesiana

Zitieren

APA

Mosquera, M., & Bojorque, R. (2026). Beyond Predictive Performance: SHAP-Based Explanations and Alignment with Literature-Based Indicators in Code Smell Detection. Information, 17(11), 1184. https://doi.org/10.3390/info17111184

Vancouver

Mosquera M, Bojorque R. Beyond Predictive Performance: SHAP-Based Explanations and Alignment with Literature-Based Indicators in Code Smell Detection. Information. 2026;17(11):1184. doi:10.3390/info17111184

BibTeX
@article{Mosquera2026,
  author = {Mosquera, Marcela and Bojorque, Rodolfo},
  title = {Beyond Predictive Performance: SHAP-Based Explanations and Alignment with Literature-Based Indicators in Code Smell Detection},
  journal = {Information},
  year = {2026},
  volume = {17},
  number = {11},
  pages = {1184},
  doi = {10.3390/info17111184}
}
Kostenlos starten →