EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Revisiting Sentiment Analysis for Software Engineering in the Era of Large Language Models

Ting Zhang, Ivana Clairine Irsan, Ferdian Thung, David Lo

ACM Transactions on Software Engineering and Methodology · 31.03.2025 · 34 (3) · 1-30

DOI: 10.1145/3697009 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 23.02.2025. Verfasst von Ting Zhang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation Singapore.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Software development involves collaborative interactions where stakeholders express opinions across various platforms. Recognizing the sentiments conveyed in these interactions is crucial for the effective development and ongoing maintenance of software systems. For software products, analyzing the sentiment of user feedback, e.g., reviews, comments, and forum posts can provide valuable insights into user satisfaction and areas for improvement. This can guide the development of future updates and features. However, accurately identifying sentiments in software engineering datasets remains challenging. This study investigates bigger large language models (bLLMs) in addressing the labeled data shortage that hampers fine-tuned smaller large language models (sLLMs) in software engineering tasks. We conduct a comprehensive empirical study using five established datasets to assess three open source bLLMs in zero-shot and few-shot scenarios. Additionally, we compare them with fine-tuned sLLMs, using sLLMs to learn contextual embeddings of text from software platforms. Our experimental findings demonstrate that bLLMs exhibit state-of-the-art performance on datasets marked by limited training data and imbalanced distributions. bLLMs can also achieve excellent performance under a zero-shot setting. However, when ample training data are available or the dataset exhibits a more balanced distribution, fine-tuned sLLMs can still achieve superior results.

Förderung

  • National Research Foundation Singapore

Zitieren

APA

Zhang, T., Irsan, I. C., Thung, F., & Lo, D. (2025). Revisiting Sentiment Analysis for Software Engineering in the Era of Large Language Models. ACM Transactions on Software Engineering and Methodology, 34(3), 1-30. https://doi.org/10.1145/3697009

Vancouver

Zhang T, Irsan IC, Thung F, Lo D. Revisiting Sentiment Analysis for Software Engineering in the Era of Large Language Models. ACM Transactions on Software Engineering and Methodology. 2025;34(3):1-30. doi:10.1145/3697009

BibTeX
@article{Zhang2025,
  author = {Zhang, Ting and Irsan, Ivana Clairine and Thung, Ferdian and Lo, David},
  title = {Revisiting Sentiment Analysis for Software Engineering in the Era of Large Language Models},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2025},
  volume = {34},
  number = {3},
  pages = {1--30},
  doi = {10.1145/3697009}
}
Kostenlos starten →