EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Fine-Tuning Arabic Large Language Models for improved multi-turn dialogue: A blueprint for synthetic data generation and benchmarking

Ahmed Mahmoud Misbah iD, Mohamed Farouk, Mustafa AbdulAzim

PLOS One · 2026

DOI: 10.1371/journal.pone.0341905 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2026. Verfasst von Ahmed Mahmoud Misbah und 2 weiteren Autor:innen. Beteiligte Einrichtung: College of Computing and Information Technology, Arab Academy for Science, Technology and Maritime Transport, Cairo, Egypt. Hauptthemen laut MEDLINE: Large Language Models, Benchmarking, Communication, Language. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The rapid evolution of Large Language Models (LLMs) has fueled increasing interest in developing Arabic conversational systems capable of sustaining coherent multi-turn dialogues. However, progress remains constrained by the scarcity of large-scale, diverse, and high-quality datasets specifically designed for Arabic multi-turn interaction. This study presents a reproducible methodology for constructing such a dataset through structured prompting of an instruction-tuned Arabic LLM (Jais-13b-chat), yielding 43,316 multi-turn conversations across 93 topics and 151 countries. Two pre-trained Arabic language models (ArabianGPT-08B-V2 and AraGPT2-mega) were fine-tuned on this synthetic data and benchmarked against multilingual instruction-tuned baselines using a comprehensive evaluation framework combining automatic metrics (Perplexity and RAVEN) with structured human evaluation. Fine-tuned ArabianGPT-08B-V2 achieved the highest RAVEN score (0.823) for cross-model comparison, outperforming both fine-tuned AraGPT2-mega and instruction-tuned baselines while maintaining strong within-model perplexity (9.4). Human evaluation by two independent raters demonstrated acceptable inter-rater reliability (Cohen’s κ = 0.229–0.739) with positive rank correlations (Spearman ρ = 0.424–0.759), yielding overall quality scores of 4.04–4.34 on a five-point scale. These findings demonstrate that high-quality, LLM-generated synthetic data effectively improves Arabic conversational models, providing a scalable, resource-efficient blueprint for dialogue systems in low-resource and culturally specific settings.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsBenchmarkingCommunicationLanguageHumans

Autor:innen und Institutionen

  • College of Computing and Information Technology, Arab Academy for Science, Technology and Maritime Transport, Cairo, Egypt.
  • College of Computing and Information Technology, Arab Academy for Science, Technology and Maritime Transport, Alexandria, Egypt.

ORCID-Profile: Ahmed Mahmoud Misbah

Zitieren

APA

Misbah, A. M., Farouk, M., & AbdulAzim, M. (2026). Fine-Tuning Arabic Large Language Models for improved multi-turn dialogue: A blueprint for synthetic data generation and benchmarking. PLOS One. https://doi.org/10.1371/journal.pone.0341905

Vancouver

Misbah AM, Farouk M, AbdulAzim M. Fine-Tuning Arabic Large Language Models for improved multi-turn dialogue: A blueprint for synthetic data generation and benchmarking. PLOS One. 2026. doi:10.1371/journal.pone.0341905

BibTeX
@article{Misbah2026,
  author = {Misbah, Ahmed Mahmoud and Farouk, Mohamed and AbdulAzim, Mustafa},
  title = {Fine-Tuning Arabic Large Language Models for improved multi-turn dialogue: A blueprint for synthetic data generation and benchmarking},
  journal = {PLOS One},
  year = {2026},
  doi = {10.1371/journal.pone.0341905}
}
Kostenlos starten →