EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Integrating Thesaurus-Based Knowledge into Transformer Models for Semantic Understanding of Domain-Specific Texts

Bayangali Abdygalym, Saule Tazhibayeva, Madina Sambetbayeva, Aigerim Yerimbetova, Roman Taberkhan, Manzura Abjalova, Aidos Sabdenov, Elmira Daiyrbayeva

Computers · 07.05.2026 · 15 (5) · 297

DOI: 10.3390/computers15050297 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computers, online erschienen am 07.05.2026. Verfasst von Bayangali Abdygalym und 7 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science and Higher Education of the Republic of Kazakhstan.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Integrating structured linguistic resources into deep learning architectures represents a key challenge in domain-oriented NLP. This study proposes a framework for incorporating knowledge from a military thesaurus of the Ground Forces, structured according to the XML Zthes standard, into pre-trained transformed language models, including KazBERT, multilingual BERT, and XLM-RoBERTA. The approach addresses two interrelated tasks in specialized terminology processing: concept linking and semantic search. Unlike existing knowledge-injection methods designed primarily for general-domain applications, this framework formalizes the mapping of Zthes elements, such as Term, Broader term, Narrower term, Related term, ScopeNote, Language, and Source into structured textual representations that can be directly processed by transformer architectures. Fine-tuning is conducted on a dataset of 18,400 training instances automatically generated from the thesaurus, including synonym pairs, hierarchical relations (hyperonymy and hyponymy), associative links, and definitional descriptions. Experimental evaluation demonstrated that thesaurus-enriched models outperform baseline architectures across all major metrics. XLM-RoBERTA model achieves F1 = 0.84 and Top-5 accuracy = 0.94 in the concept linking task, representing a five-point improvement over the baseline. The model reaches Macro-F1 = 0.84 across four relation types. Results obtained on a specialized test set derived from terminology databases of Kazakhstan’s Armed Forces confirm robust cross-lingual generalization across Kazakh, Russian and English military discourse.

Förderung

  • Ministry of Science and Higher Education of the Republic of Kazakhstan

Zitieren

APA

Abdygalym, B., Tazhibayeva, S., Sambetbayeva, M., Yerimbetova, A., Taberkhan, R., Abjalova, M., Sabdenov, A., & Daiyrbayeva, E. (2026). Integrating Thesaurus-Based Knowledge into Transformer Models for Semantic Understanding of Domain-Specific Texts. Computers, 15(5), 297. https://doi.org/10.3390/computers15050297

Vancouver

Abdygalym B, Tazhibayeva S, Sambetbayeva M, Yerimbetova A, Taberkhan R, Abjalova M, et al. Integrating Thesaurus-Based Knowledge into Transformer Models for Semantic Understanding of Domain-Specific Texts. Computers. 2026;15(5):297. doi:10.3390/computers15050297

BibTeX
@article{Abdygalym2026,
  author = {Abdygalym, Bayangali and Tazhibayeva, Saule and Sambetbayeva, Madina and Yerimbetova, Aigerim and Taberkhan, Roman and Abjalova, Manzura and Sabdenov, Aidos and Daiyrbayeva, Elmira},
  title = {Integrating Thesaurus-Based Knowledge into Transformer Models for Semantic Understanding of Domain-Specific Texts},
  journal = {Computers},
  year = {2026},
  volume = {15},
  number = {5},
  pages = {297},
  doi = {10.3390/computers15050297}
}
Kostenlos starten →