MAPFusion: Enhancing RoBERTa for News Classification with Multi-Head Attention Pooling and Feature Fusion
International Scientific Technical and Economic Research · 16.11.2025 · 3 (4) · 71-84
DOI: 10.71451/istaer2555 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Scientific Technical and Economic Research, online erschienen am 16.11.2025. Verfasst von Xinran Bu. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 12 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This paper proposes MAPFusion, a novel approach to enhance RoBERTa for news text classification using multi-head attention pooling. Traditional methods often rely on static pooling or simple averaging of [CLS] tokens, which can overlook subtle contextual information across token positions. The proposed method addresses this limitation with multi-head attention pooling (MAP), which captures context-aware representations by aggregating token-level embeddings with learned attention weights across multiple representation subspaces. The outputs from multiple attention heads are then integrated through a feature fusion layer to create a comprehensive sentence representation. This component is seamlessly integrated into the output layer of RoBERTa, preserving its pre-trained weights and requiring minimal additional parameters during fine-tuning. Experiments on benchmark datasets demonstrate that MAP-Fusion consistently outperforms baseline models, achieving significant improvements in classification accuracy. The framework is computationally efficient, broadly applicable to various text classification tasks, and provides a principled approach to more effectively utilize the latent representations of RoBERTa. Our work highlights the importance of adaptive feature aggregation in Transformer-based models, providing insights for future representation learning research. The code for this paper is available by contacting the respective authors.
Zitieren
APA
Bu, X. (2025). MAPFusion: Enhancing RoBERTa for News Classification with Multi-Head Attention Pooling and Feature Fusion. International Scientific Technical and Economic Research, 3(4), 71-84. https://doi.org/10.71451/istaer2555
Vancouver
Bu X. MAPFusion: Enhancing RoBERTa for News Classification with Multi-Head Attention Pooling and Feature Fusion. International Scientific Technical and Economic Research. 2025;3(4):71-84. doi:10.71451/istaer2555
BibTeX
@article{Bu2025,
author = {Bu, Xinran},
title = {MAPFusion: Enhancing RoBERTa for News Classification with Multi-Head Attention Pooling and Feature Fusion},
journal = {International Scientific Technical and Economic Research},
year = {2025},
volume = {3},
number = {4},
pages = {71--84},
doi = {10.71451/istaer2555}
}
EUVIMED