EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Behavioral Analysis of Semantic Similarity Metrics under Transformer-Based Representations

Musthofa Galih Pradana, Nindy Irzavika, Nurhuda Maulana, Syaila Ananta Karenina, Salma Ashiila Rabbani

Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · 20.04.2026 · 10 (2) · 332-340

DOI: 10.29207/resti.v10i2.7490 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), online erschienen am 20.04.2026. Verfasst von Musthofa Galih Pradana und 4 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Knowledge extraction has several approaches such as traditional approaches that rely on lexical representation capabilities, one of which is TF-IDF whose implementation can be combined with several classic similarity metrics such as cosine similarity and dice coefficient similarity. In addition to applying the lexical representation approach, this study tries to apply it to a more modern type of representation, namely transformer-embedding-based contextual representation. The data used in this study is abstract document data of students' theses. The findings of the study show that contextual embedding changes the behavior of similarity values. The results of the analysis showed an average ranking shift of 6.70 positions. The test results showed a weak rating correlation value (Spearman = 0.22; Kendall = 0.146), and the high-ranking alignment measured with NDCG (0.97), which shows structural differences in the order of similarity between lexical and contextual representations. Other findings also show that the gap in ranking produced by the two representations used is quite far due to the difference in the mechanism and working pattern of the two representations that are far different, the selection of the type of representation must be on the characteristics of the data to be processed, if looking at the character of the text data in academic documents, the selection of contextual representations based on transformer embedding will be more suitable with contextual understanding to avoid the use of variations in words avoid plagiarism detection when applying a semantic-based representation approach.

Zitieren

APA

Pradana, M. G., Irzavika, N., Maulana, N., Karenina, S. A., & Rabbani, S. A. (2026). Behavioral Analysis of Semantic Similarity Metrics under Transformer-Based Representations. Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), 10(2), 332-340. https://doi.org/10.29207/resti.v10i2.7490

Vancouver

Pradana MG, Irzavika N, Maulana N, Karenina SA, Rabbani SA. Behavioral Analysis of Semantic Similarity Metrics under Transformer-Based Representations. Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi). 2026;10(2):332-340. doi:10.29207/resti.v10i2.7490

BibTeX
@article{Pradana2026,
  author = {Pradana, Musthofa Galih and Irzavika, Nindy and Maulana, Nurhuda and Karenina, Syaila Ananta and Rabbani, Salma Ashiila},
  title = {Behavioral Analysis of Semantic Similarity Metrics under Transformer-Based Representations},
  journal = {Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)},
  year = {2026},
  volume = {10},
  number = {2},
  pages = {332--340},
  doi = {10.29207/resti.v10i2.7490}
}
Kostenlos starten →