EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Validation of a large language models-based socio-political sentiment analysis system with a retrieval-augmented generation approach

Sidharta Sidharta, Hady Pranoto, Frederik Gasa, Nur Kholis, Ardvin Kester S Ong

International Journal of Advances in Intelligent Informatics · 31.08.2026 · 12 (3) · 815

DOI: 10.26555/ijain.v12i3.2389 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Advances in Intelligent Informatics, online erschienen am 31.08.2026. Verfasst von Sidharta Sidharta und 4 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Socio-political discourse on digital platforms generates large volumes of unstructured textual data, posing challenges for reliable sentiment analysis and contextual interpretation. Conventional sentiment analysis approaches often lack contextual grounding, while direct use of Large Language Models (LLMs) may introduce hallucinations, bias, and excessive generalization. To address these issues, this study proposes an LLM-based sentiment analysis system that integrates multi-source data acquisition, zero-shot sentiment classification, Retrieval-Augmented Generation (RAG), and generative reasoning for contextual interpretation. Unlike prior studies that mainly focus on improving model accuracy, this research emphasizes system-level validation of the entire analytical pipeline. The evaluation includes sentiment calibration using a Golden Dataset, data ingestion performance analysis, retrieval quality assessment based on semantic distance, and generative evaluation using an LLM-as-a-Judge framework. Experimental results indicate that the system provides stable data acquisition and produces coherent analytical reports. However, retrieval noise and neutrality bias introduced by LLM alignment mechanisms still affect the groundedness of generated outputs. These findings highlight the importance of retrieval quality and demonstrate the potential of LLMs as reasoning and evaluation components for analyzing complex socio-political discourse.

Zitieren

APA

Sidharta, S., Pranoto, H., Gasa, F., Kholis, N., & Ong, A. K. S. (2026). Validation of a large language models-based socio-political sentiment analysis system with a retrieval-augmented generation approach. International Journal of Advances in Intelligent Informatics, 12(3), 815. https://doi.org/10.26555/ijain.v12i3.2389

Vancouver

Sidharta S, Pranoto H, Gasa F, Kholis N, Ong AKS. Validation of a large language models-based socio-political sentiment analysis system with a retrieval-augmented generation approach. International Journal of Advances in Intelligent Informatics. 2026;12(3):815. doi:10.26555/ijain.v12i3.2389

BibTeX
@article{Sidharta2026,
  author = {Sidharta, Sidharta and Pranoto, Hady and Gasa, Frederik and Kholis, Nur and Ong, Ardvin Kester S},
  title = {Validation of a large language models-based socio-political sentiment analysis system with a retrieval-augmented generation approach},
  journal = {International Journal of Advances in Intelligent Informatics},
  year = {2026},
  volume = {12},
  number = {3},
  pages = {815},
  doi = {10.26555/ijain.v12i3.2389}
}
Kostenlos starten →