EUVIMEDEUVIMED entdecken
Crossref

Are prompts all you need? Evaluating prompt-based Large Language Models (LLM)s for software requirements classification

Manal Binkhonain, Reem Alfayez

Requirements Engineering · 2025-12 · 30 (4) · 423-443

DOI: 10.1007/s00766-025-00451-8 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Requirements Engineering, online erschienen am 16.09.2025. Verfasst von Manal Binkhonain und 1 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 48 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Context

Requirements classification is a fundamental task in software engineering that involves classifying natural language requirements into predefined classes, such as functional and non-functional requirements. Accurate classification is critical to the success of software projects, as it helps reduce development risks and improve overall software quality. However, most existing classification models rely on supervised learning approaches, which demand large, annotated datasets that are resource-intensive, time-consuming, and dependent on domain-specific expertise to acquire. Moreover, these models often struggle with generalizability, as they typically require retraining or fine-tuning for each new classification task.

Objective

This study aims to address the challenge of dataset scarcity in requirements classification by investigating the effectiveness of prompt-based LLMs.

Method

We conducted an empirical benchmarking study to evaluate and statistically compare the performance of multiple prompt-based LLMs and prompting techniques across several requirements classification tasks on two English requirements classification benchmark datasets: PROMISE and SecReq. The evaluation incorporated four prompting techniques: zero-shot, few-shot, persona, and chain of thought (CoT) prompts. For each task, we assessed and statistically compared the performance of different model–prompt configurations. We then compared the best-performing prompt-based LLM configurations on each task against the performance of the state-of-the-art fine-tuned transformer-based model.

Results

The results demonstrate that prompt-based LLMs, particularly when paired with few-shot prompting, can achieve classification performance comparable to or even exceeding that of the state-of-the-art fine-tuned transformer-based model. Furthermore, augmenting few-shot prompts with either persona alone or a combination of persona and CoT can potentially improve model performance.

Conclusion

This study highlights the potential of prompt-based LLMs as a practical and scalable alternative to traditional supervised approaches for requirements classification. By reducing the dependency on large annotated datasets, these models provide a flexible solution to overcoming data scarcity and enhancing the generalizability of automated requirements classification tools.

Zitieren

APA

Binkhonain, M., & Alfayez, R. (2025). Are prompts all you need? Evaluating prompt-based Large Language Models (LLM)s for software requirements classification. Requirements Engineering, 30(4), 423-443. https://doi.org/10.1007/s00766-025-00451-8

Vancouver

Binkhonain M, Alfayez R. Are prompts all you need? Evaluating prompt-based Large Language Models (LLM)s for software requirements classification. Requirements Engineering. 2025;30(4):423-443. doi:10.1007/s00766-025-00451-8

BibTeX
@article{Binkhonain2025,
  author = {Binkhonain, Manal and Alfayez, Reem},
  title = {Are prompts all you need? Evaluating prompt-based Large Language Models (LLM)s for software requirements classification},
  journal = {Requirements Engineering},
  year = {2025},
  volume = {30},
  number = {4},
  pages = {423--443},
  doi = {10.1007/s00766-025-00451-8}
}
Kostenlos starten →