Sentence Embedding Using Multimodal Approach: Combining FastText with AraBERT for Arabic Text Representation
F1000Research · 06.02.2026 · 15 · 206
DOI: 10.12688/f1000research.174830.1 ↗Kurz gefasst
Fachartikel in der Zeitschrift F1000Research, online erschienen am 06.02.2026. Verfasst von Hind Almayyali und 1 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Sentence-embedding models transform sentences into dense vector representations that capture their semantic meanings. These representations enable deep learning to perform many tasks efficiently, such as similarity measurement, retrieval, and summarization, with improved semantic understanding. Existing sentence embedding models often struggle to capture the semantic richness and morphological complexity of Arabic, limiting their effectiveness in tasks such as semantic similarity, question answering, summarization, and information retrieval.
Objectives
This study aims to develop a novel sentence-embedding framework tailored for Arabic that addresses the shortcomings of current models by integrating contextual and linguistic features.
Methods
We propose a multimodal architecture that combines a fine-tuned Sentence-AraBERT (SAraBERT) model with pre-trained FastText embeddings. The model is evaluated on standard Arabic Semantic Textual Similarity (STS) benchmarks using the Mean Squared Error (MSE) and Pearson Correlation Coefficient.
Results
Experimental results show that the proposed model outperforms existing baselines, achieving lower MSE values (0.0355) and higher correlation scores (0.8053), indicating a stronger alignment with human-annotated similarity judgments on the ATrD dataset.
Conclusion
The findings demonstrate the effectiveness of multimodal SAraBERT-based embeddings in enhancing sentence-level semantic understanding of Arabic. This study advances Natural Language Processing (NLP) capabilities for underrepresented languages and provides a foundation for future research on Arabic language understanding using deep learning techniques.
Zitieren
APA
Almayyali, H., & Aliwy, A. (2026). Sentence Embedding Using Multimodal Approach: Combining FastText with AraBERT for Arabic Text Representation. F1000Research, 15, 206. https://doi.org/10.12688/f1000research.174830.1
Vancouver
Almayyali H, Aliwy A. Sentence Embedding Using Multimodal Approach: Combining FastText with AraBERT for Arabic Text Representation. F1000Research. 2026;15:206. doi:10.12688/f1000research.174830.1
BibTeX
@article{Almayyali2026,
author = {Almayyali, Hind and Aliwy, Ahmed},
title = {Sentence Embedding Using Multimodal Approach: Combining FastText with AraBERT for Arabic Text Representation},
journal = {F1000Research},
year = {2026},
volume = {15},
pages = {206},
doi = {10.12688/f1000research.174830.1}
}
EUVIMED