EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

SanskritBERT: Language-Specific Transformer Modelling for Classical Sanskrit Texts

Soumya Sharma, Tanuj Saxena, Kusum Lata

DMPedia Lecture Notes in Multidisciplinary Research · 15.03.2026 · 1313-1323

DOI: 10.65890/dmp.lnmr.impact26.170 ↗

Kurz gefasst

Fachartikel in der Zeitschrift DMPedia Lecture Notes in Multidisciplinary Research, online erschienen am 15.03.2026. Verfasst von Soumya Sharma und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent breakthroughs in transformer-based architectures, such as BERT, have revolutionised natural language processing (NLP) across many languages. However, low-resource and morphologically complex languages like Sanskrit remain poorly represented in large-scale pretrained models due to scarce digital corpora, orthographic variation, and compounding. This paper presents a fully custom Sanskrit BERT model trained from scratch on a corpus of 21 million+ curated Sanskrit sentences written in pure Devanagari script. To represent the morphological richness of the language, a Sentence-Piece Unigram tokeniser with a 64k subword vocabulary was built, and a light 6-layer BERT architecture with 256-dimensional hidden states was used to balance performance and compute efficiency. Experimental results show that the model significantly outperforms multilingual baselines such as mBERT, IndicBERT, and MuRIL on masked language modelling test sets, achieving Top-1 accuracy of 0.35, Top-5 accuracy of 0.50, and a perplexity score of 69.0. These outcomes confirm the merits of corpus-specific tokenisation and pretraining in monolingual style for morphologically rich classical languages. Future work will investigate scaling the model to larger architectures, incorporating more complex subword representations, and fine-tuning for Sanskrit NLP downstream tasks such as word segmentation, translation, and semantic role labelling.

Zitieren

APA

Sharma, S., Saxena, T., & Lata, K. (2026). SanskritBERT: Language-Specific Transformer Modelling for Classical Sanskrit Texts. DMPedia Lecture Notes in Multidisciplinary Research, 1313-1323. https://doi.org/10.65890/dmp.lnmr.impact26.170

Vancouver

Sharma S, Saxena T, Lata K. SanskritBERT: Language-Specific Transformer Modelling for Classical Sanskrit Texts. DMPedia Lecture Notes in Multidisciplinary Research. 2026:1313-1323. doi:10.65890/dmp.lnmr.impact26.170

BibTeX
@article{Sharma2026,
  author = {Sharma, Soumya and Saxena, Tanuj and Lata, Kusum},
  title = {SanskritBERT: Language-Specific Transformer Modelling for Classical Sanskrit Texts},
  journal = {DMPedia Lecture Notes in Multidisciplinary Research},
  year = {2026},
  pages = {1313--1323},
  doi = {10.65890/dmp.lnmr.impact26.170}
}
Kostenlos starten →