Can LLMs Pass the UGC-NET in LIS? Benchmarking RAG-Enhanced Accuracy Against Baseline Performance
DESIDOC Journal of Library & Information Technology · 02.07.2026 · 46 (4) · 363-374
DOI: 10.14429/djlit.21932 ↗Kurz gefasst
Fachartikel in der Zeitschrift DESIDOC Journal of Library & Information Technology, online erschienen am 02.07.2026. Verfasst von Madhumita Neogi und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 2.5).
Abstract (Originalsprache)
Academia raises concerns in numerous research studies about the accuracy of generative AI in discipline-specific question-answering systems, particularly large language models (LLMs). This study evaluates the accuracy of seven state-of-the-art LLMs, including an indigenous model, in library and information science (LIS) by using a dataset of 1,100 questions from the UGC-NET (LIS) examination (2019-2025). It proposes an experimental design to compare “baseline” performance of selected LLMs (pre-trained models as available in the public domain) against a Retrieval-Augmented Generation (RAG) system, configured locally and trained with domain-specific Open Educational Resources (OERs) in two phases: first using standard IGNOU courseware (Phase I), and second using an extended corpus containing Indian library legislation, libraries in India, encyclopedic resources on LIS and other standardised learning resources (Phase II). The results indicate that while baseline models suffer from knowledge gaps regarding Indian contexts and subject depth, RAG provides a consistent performance boost for all LLMs selected in this study. The final result reveals that Gemini 2.5 Pro led the performance across all levels (baseline, Phase I, and Phase II), achieving the highest accuracy of 97.27 % in the final RAG phase, while GPT-OSS-120b led in error reduction (63.26 %) and Sarvam-M (India) secured the largest absolute gain of +18.18 percentage points (from baseline to the final phase). The findings indicate that smaller, sovereign models augmented with curated, standard, domain-specific learning resources can compete with global giants. This framework offers a viable pathway for trustworthy and economic AI adoption in LIS education.
Zitieren
APA
Neogi, M., & Mukhopadhyay, P. (2026). Can LLMs Pass the UGC-NET in LIS? Benchmarking RAG-Enhanced Accuracy Against Baseline Performance. DESIDOC Journal of Library & Information Technology, 46(4), 363-374. https://doi.org/10.14429/djlit.21932
Vancouver
Neogi M, Mukhopadhyay P. Can LLMs Pass the UGC-NET in LIS? Benchmarking RAG-Enhanced Accuracy Against Baseline Performance. DESIDOC Journal of Library & Information Technology. 2026;46(4):363-374. doi:10.14429/djlit.21932
BibTeX
@article{Neogi2026,
author = {Neogi, Madhumita and Mukhopadhyay, Parthasarathi},
title = {Can LLMs Pass the UGC-NET in LIS? Benchmarking RAG-Enhanced Accuracy Against Baseline Performance},
journal = {DESIDOC Journal of Library & Information Technology},
year = {2026},
volume = {46},
number = {4},
pages = {363--374},
doi = {10.14429/djlit.21932}
}
EUVIMED