Refining keyphrase extraction in academic articles via Bi-LSTM and multi-head attention
Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology · 18.11.2026
DOI: 10.1177/18758967261509211 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology, online erschienen am 18.11.2026. Verfasst von Abdellah El Omari und 1 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben.
Abstract (Originalsprache)
Keyphrase extraction from academic articles is an important step toward better information retrieval, indexing, and summarization in large scholarly databases. Traditional approaches, such as TF-IDF and TextRank, achieve limited success in capturing complex linguistic patterns, while more recent neural models, such as Bi-LSTM-CRF, still struggle with long-range dependencies. In this work, we propose a Bi-LSTM model with Multi-head Attention, enhanced by a custom word embedding trained on a large ArXiv corpus, and evaluate it exclusively on the abstracts of scientific articles, since abstracts condense a paper's core information and reduce noise. The model was trained on KP20 K and evaluated, at token and phrase level, across five additional corpora (ArXiv, Krapivin, Inspec, PubMed, SemEval2010), obtaining a token-level macro F1-score between 28% and 44% depending on the corpus, with no clear in-domain advantage for KP20 K over some out-of-domain corpora. We complement this evaluation with a focused ablation isolating the contribution of Multi-head Attention, preprocessing, embedding source, and class weighting, and with an empirical comparison against three fine-tuned Transformer encoders (SciBERT, BioBERT, DeBERTa). Domain-adapted Transformers substantially outperform our lighter architecture on both token- and phrase-level metrics, while DeBERTa's fine-tuning collapsed to majority-class prediction under our protocol. The contribution of this study thus lies in cross-corpus evaluation and generalization analysis rather than in state-of-the-art performance. Taken together, these experiments clarify, more than they resolve, the trade-offs between lightweight recurrent architectures and modern Transformer encoders for scientific keyphrase extraction, and the domain-generalization challenges that remain open in this task.
Zitieren
APA
Omari, A. E., & Antari, J. (2026). Refining keyphrase extraction in academic articles via Bi-LSTM and multi-head attention. Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology. https://doi.org/10.1177/18758967261509211
Vancouver
Omari AE, Antari J. Refining keyphrase extraction in academic articles via Bi-LSTM and multi-head attention. Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology. 2026. doi:10.1177/18758967261509211
BibTeX
@article{Omari2026,
author = {Omari, Abdellah El and Antari, Jilali},
title = {Refining keyphrase extraction in academic articles via Bi-LSTM and multi-head attention},
journal = {Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology},
year = {2026},
doi = {10.1177/18758967261509211}
}
EUVIMED