A Deep Learning Approach to Classifying Software Requirements: The Application of Transformer-Based Ensemble Learning and Attention-Based Fusion
Foundations of Computing and Decision Sciences · 01.03.2026 · 51 (1) · 71-110
DOI: 10.2478/fcds-2026-0003 ↗Kurz gefasst
Fachartikel in der Zeitschrift Foundations of Computing and Decision Sciences, online erschienen am 17.03.2026. Verfasst von Azaz Ahmed Kiani. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 3.0).
Abstract (Originalsprache)
The functional requirements (FRs) classification in software requirements classification (SRC) is a difficult task due to class imbalance, fine-grained subcategories, and semantic complexities. Existing Machine Learning (ML) and Deep Learning (DL) models often rely on handcrafted characteristics or overlook the contextual meaning. This work presents a novel hybrid ensemble framework that refines three pre-trained transformers (BERT, DistilBERT, and RoBERTa) and combines them using two mechanisms: (1) an Attention-Based Fusion Mechanism that dynamically selects the most contextually relevant transformer for each instance, and (2) an Accuracy-Per-Class Weighted Ensemble that assigns weights based on per-class validation accuracy. Tested on multiple datasets, the approach outperformed single-transformer and DL models (CNN, LSTM, BiLSTM, and GRU) by a large margin (p < 0.001), achieving 95% accuracy and 0.94 F1-score. To the best of our knowledge, this is the first study to combine attention fusion and transformer-based ensembles for SRC, establishing a new standard for SRC.
Zitieren
APA
Kiani, A. A. (2026). A Deep Learning Approach to Classifying Software Requirements: The Application of Transformer-Based Ensemble Learning and Attention-Based Fusion. Foundations of Computing and Decision Sciences, 51(1), 71-110. https://doi.org/10.2478/fcds-2026-0003
Vancouver
Kiani AA. A Deep Learning Approach to Classifying Software Requirements: The Application of Transformer-Based Ensemble Learning and Attention-Based Fusion. Foundations of Computing and Decision Sciences. 2026;51(1):71-110. doi:10.2478/fcds-2026-0003
BibTeX
@article{Kiani2026,
author = {Kiani, Azaz Ahmed},
title = {A Deep Learning Approach to Classifying Software Requirements: The Application of Transformer-Based Ensemble Learning and Attention-Based Fusion},
journal = {Foundations of Computing and Decision Sciences},
year = {2026},
volume = {51},
number = {1},
pages = {71--110},
doi = {10.2478/fcds-2026-0003}
}
EUVIMED