EUVIMEDEUVIMED entdecken
Crossref

Large language model ChatGPT versus small deep learning models for self‐admitted technical debt detection: Why not together?

Jun Li, Lixian Li, Jin Liu, Xiao Yu, Xiao Liu, Jacky Wai Keung

Software: Practice and Experience · 2025-01 · 55 (1) · 3-28

DOI: 10.1002/spe.3360 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Software: Practice and Experience, online erschienen am 28.06.2024. Verfasst von Jun Li und 5 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 85 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Given the increasing complexity and volume of Self‐Admitted Technical Debts (SATDs), how to efficiently detect them becomes critical in software engineering practice for improving code quality and project efficiency. Although current deep learning methods have achieved good performance in detecting SATDs in code comments, they lack explanation. Large language models such as ChatGPT are increasingly being applied to text classification tasks due to their ability to provide explanations for classification results, but it is unclear how effective ChatGPT is for SATD classification. As the first in‐depth study of ChatGPT for SATD detection, we evaluate ChatGPT's effectiveness, compare it with small deep learning models, and find that ChatGPT performs better on Recall, while small models perform better on Precision. Furthermore, to enhance the performance of these approaches, we propose a novel fusion approach named FSATD which combines ChatGPT with small models for SATD detection so as to provide reliable explanations. Through extensive experiments on 62,276 comments from 10 open‐source projects, we show that FSATD outperforms existing methods in performance of F1‐score in cross‐project scenarios. Additionally, FSATD allows for flexible adjustment of fusion strategies, adapting to different requirements of various application scenarios, and can achieve the best Precision, Recall, or F1‐score.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Li, J., Li, L., Liu, J., Yu, X., Liu, X., & Keung, J. W. (2025). Large language model ChatGPT versus small deep learning models for self‐admitted technical debt detection: Why not together?. Software: Practice and Experience, 55(1), 3-28. https://doi.org/10.1002/spe.3360

Vancouver

Li J, Li L, Liu J, Yu X, Liu X, Keung JW. Large language model ChatGPT versus small deep learning models for self‐admitted technical debt detection: Why not together?. Software: Practice and Experience. 2025;55(1):3-28. doi:10.1002/spe.3360

BibTeX
@article{Li2025,
  author = {Li, Jun and Li, Lixian and Liu, Jin and Yu, Xiao and Liu, Xiao and Keung, Jacky Wai},
  title = {Large language model ChatGPT versus small deep learning models for self‐admitted technical debt detection: Why not together?},
  journal = {Software: Practice and Experience},
  year = {2025},
  volume = {55},
  number = {1},
  pages = {3--28},
  doi = {10.1002/spe.3360}
}
Kostenlos starten →