EUVIMEDEUVIMED entdecken
Crossref

Diffusion Recommender Models and the Illusion of Progress: A Concerning Study of Reproducibility and a Conceptual Mismatch

Michael Benigni, Maurizio Ferrari Dacrema, Dietmar Jannach

ACM Transactions on Recommender Systems · 30.09.2026 · 4 (3) · 1-69

DOI: 10.1145/3795792 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Recommender Systems, online erschienen am 16.03.2026. Verfasst von Michael Benigni und 2 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 81 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Countless new machine learning models are published every year and are reported to significantly advance the state-of-the-art in top-n recommendation. However, earlier reproducibility studies indicate that progress in this area may be quite limited, due to widespread methodological issues, e.g., comparisons with untuned baseline models, creating an illusion of progress . In this work, we examine whether these problems persist in today’s research by attempting to reproduce nine SIGIR 2023 and 2024 recommendation algorithms based on Denoising Diffusion Probabilistic Models, a recent but rapidly expanding research area. Only 25% of reported results are fully reproducible and, since the original papers relied on weak baselines , they do not establish the superiority of diffusion models over state-of-the-art methods. In our controlled evaluations, well-tuned simpler baselines consistently exceed the diffusion-based models’ effectiveness reported in the original papers. Furthermore, we identify key mismatches between the characteristics of diffusion models and those of the traditional top-n recommendation task, raising doubts about their suitability for recommendation. Moreover, in the analyzed papers, the generative capabilities of these models are constrained to a minimum. Overall, our results call for greater scientific rigor and a disruptive change in the research and publication culture in this area.

Zitieren

APA

Benigni, M., Dacrema, M. F., & Jannach, D. (2026). Diffusion Recommender Models and the Illusion of Progress: A Concerning Study of Reproducibility and a Conceptual Mismatch. ACM Transactions on Recommender Systems, 4(3), 1-69. https://doi.org/10.1145/3795792

Vancouver

Benigni M, Dacrema MF, Jannach D. Diffusion Recommender Models and the Illusion of Progress: A Concerning Study of Reproducibility and a Conceptual Mismatch. ACM Transactions on Recommender Systems. 2026;4(3):1-69. doi:10.1145/3795792

BibTeX
@article{Benigni2026,
  author = {Benigni, Michael and Dacrema, Maurizio Ferrari and Jannach, Dietmar},
  title = {Diffusion Recommender Models and the Illusion of Progress: A Concerning Study of Reproducibility and a Conceptual Mismatch},
  journal = {ACM Transactions on Recommender Systems},
  year = {2026},
  volume = {4},
  number = {3},
  pages = {1--69},
  doi = {10.1145/3795792}
}
Kostenlos starten →