EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Benchmarking retrieval augmented generation LLMs for Arabic noise robustness

Saleh Almohaimeed, Abdulrahman Alabduljabbar, Mousa Jari, Mohammed Alkhowaiter, Saad Almohaimeed, Mohamad Mahmoud Al Rahhal

Frontiers in Big Data · 17.08.2026 · 9

DOI: 10.3389/fdata.2026.1884673 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Big Data, online erschienen am 17.08.2026. Verfasst von Saleh Almohaimeed und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Applied Computer Science, College of Applied Computer Science, King Saud University, Riyadh, Saudi Arabia. Die Arbeit nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Saud University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Hallucination has become a serious concern in large language models (LLMs), as these models can generate useful yet incorrect or misleading information, which has led to growing research interest in retrieval-augmented generation (RAG) as a mitigation approach. RAG provides LLMs with access to external information, such as databases or documents, which can help them to answer users' questions. Currently, several benchmarks released measure RAG performance on various LLMs; however, evaluations of the noise robustness ability in Arabic are absent. In this paper, we systematically investigated the capabilities of state-of-the-art multilingual LLMs with regard to two essential RAG abilities, noise robustness and negative rejection. To accomplish this, we generated an Arabic benchmark consisting of 300 questions along with 6,196 documents. Then, we assessed the performance of six LLMs in relation to the two aforementioned RAG abilities. The results reveal that all six LLMs were negatively affected when the noise ratio in the external documents was increased. Under the highest noise settings at 80%, the best LLM performance was for Claude-4 sonnet, in which their performance decreased by only 4.67 percentage points. Furthermore, when it comes to the negative rejection task, there has been a significant impact on all six models. The best two models, Claude-4 sonnet and Llama-4, scored 90.67% and 85.67%, respectively, while smaller models, like GPT-3.5, scored 69.33%. Furthermore, our manual analysis reveals that many errors made by LLMs are attributed to over-caution behavior. LLMs often decline to respond probably due to training mechanisms designed to reduce hallucinations. Additionally, other errors occur when there is a high lexical similarity between the question and the words of noisy documents, which causes the model to rely on irrelevant content instead of the correct information.

Schlagwörter

Stichwörter der Autor:innen

Arabic natural language processingLLM noise robustnesslarge language modelsquestion answering systemsretrieval-augmented generation

Autor:innen und Institutionen

  • Department of Applied Computer Science, College of Applied Computer Science, King Saud University, Riyadh, Saudi Arabia.
  • Department of Computer Engineering, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, Al-Kharj, Saudi Arabia.
  • Department of Digital Transformation, Institute of Public Administration, Riyadh, Saudi Arabia.

Förderung

  • King Saud University

Zitieren

APA

Almohaimeed, S., Alabduljabbar, A., Jari, M., Alkhowaiter, M., Almohaimeed, S., & Al Rahhal, M. M. (2026). Benchmarking retrieval augmented generation LLMs for Arabic noise robustness. Frontiers in Big Data, 9. https://doi.org/10.3389/fdata.2026.1884673

Vancouver

Almohaimeed S, Alabduljabbar A, Jari M, Alkhowaiter M, Almohaimeed S, Al Rahhal MM. Benchmarking retrieval augmented generation LLMs for Arabic noise robustness. Frontiers in Big Data. 2026;9. doi:10.3389/fdata.2026.1884673

BibTeX
@article{Almohaimeed2026,
  author = {Almohaimeed, Saleh and Alabduljabbar, Abdulrahman and Jari, Mousa and Alkhowaiter, Mohammed and Almohaimeed, Saad and Al Rahhal, Mohamad Mahmoud},
  title = {Benchmarking retrieval augmented generation LLMs for Arabic noise robustness},
  journal = {Frontiers in Big Data},
  year = {2026},
  volume = {9},
  doi = {10.3389/fdata.2026.1884673}
}
Kostenlos starten →