HOW ACCURATELY DO ARTİFİCİAL INTELLİGENCE CHATBOTS EXPLAİN INTERNATİONAL PROTECTİON? A COMPARATİVE EVALUATİON
Göç Araştırmaları Dergisi · 06.08.2026 · 12 (1 (Göç ve Yapay Zekâ)) · 135-149
DOI: 10.70972/gad.1887796 ↗Kurz gefasst
Fachartikel in der Zeitschrift Göç Araştırmaları Dergisi, online erschienen am 06.08.2026. Verfasst von Hamza Aydemir und 2 weiteren Autor:innen. Die Arbeit nennt 18 Literaturangaben.
Abstract (Originalsprache)
Background
This study aims to comparatively evaluate the performance of artificial intelligence chatbots in explaining fundamental concepts and processes related to international protection status. Within the scope of the research, a set of 10 questions related to international protection status was posed to the OpenAI ChatGPT-5, Google Gemini 2.5 Flash, and Microsoft Copilot Think Deeper models in October 2025.
Objective
The initial responses generated by the models were scored by five academics specialising in migration using a blind evaluation method. The evaluation process was based on the Handbook on Procedures and Criteria for Determining Refugee Status and Guidelines on International Protection developed by the United Nations High Commissioner for Refugees (UNHCR).
Method
The responses were evaluated on a five-point scale according to their accuracy and comprehensiveness. Kendall's W coefficient was used to determine inter-rater agreement in the analysis of the data obtained, while the Friedman test was used to test the differences between the artificial intelligence models.
Results
The findings show that all models are generally accurate but often provide limited and superficial explanations. It was found that ChatGPT's average scores were higher than those of other models, but there were significant differences between models, particularly in concepts open to legal interpretation, such as “complementary protection”.
Conclusion
Consequently, it was concluded that artificial intelligence chatbots could serve as auxiliary tools providing rapid access to fundamental information in the field of international protection, but could not replace expert assessment in rights-based and legally consequential individual application processes.
Zitieren
APA
Aydemir, H., Esen, İ., & Hasgül, E. (2026). HOW ACCURATELY DO ARTİFİCİAL INTELLİGENCE CHATBOTS EXPLAİN INTERNATİONAL PROTECTİON? A COMPARATİVE EVALUATİON. Göç Araştırmaları Dergisi, 12(1 (Göç ve Yapay Zekâ)), 135-149. https://doi.org/10.70972/gad.1887796
Vancouver
Aydemir H, Esen İ, Hasgül E. HOW ACCURATELY DO ARTİFİCİAL INTELLİGENCE CHATBOTS EXPLAİN INTERNATİONAL PROTECTİON? A COMPARATİVE EVALUATİON. Göç Araştırmaları Dergisi. 2026;12(1 (Göç ve Yapay Zekâ)):135-149. doi:10.70972/gad.1887796
BibTeX
@article{Aydemir2026,
author = {Aydemir, Hamza and Esen, İhsan and Hasgül, Ergün},
title = {HOW ACCURATELY DO ARTİFİCİAL INTELLİGENCE CHATBOTS EXPLAİN INTERNATİONAL PROTECTİON? A COMPARATİVE EVALUATİON},
journal = {Göç Araştırmaları Dergisi},
year = {2026},
volume = {12},
number = {1 (Göç ve Yapay Zekâ)},
pages = {135--149},
doi = {10.70972/gad.1887796}
}
EUVIMED