Comparing and assessing four AI chatbots’ competence in economics
PLOS ONE · 2024
DOI: 10.1371/journal.pone.0297804 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift PLOS ONE, 2024. Verfasst von Patrik T Hultberg und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Economics and Business, Kalamazoo College, Kalamazoo, Michigan, United States of America. Hauptthemen laut MEDLINE: Artificial Intelligence. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Artificial Intelligence (AI) chatbots have emerged as powerful tools in modern academic endeavors, presenting both opportunities and challenges in the learning landscape. They can provide content information and analysis across most academic disciplines, but significant differences exist in terms of response accuracy for conclusions and explanations, as well as word counts. This study explores four distinct AI chatbots, GPT-3.5, GPT-4, Bard, and LLaMA 2, for accuracy of conclusions and quality of explanations in the context of university-level economics. Leveraging Bloom’s taxonomy of cognitive learning complexity as a guiding framework, the study confronts the four AI chatbots with a standard test for university-level understanding of economics, as well as more advanced economics problems. The null hypothesis that all AI chatbots perform equally well on prompts that explore understanding of economics is rejected. The results are that significant differences are observed across the four AI chatbots, and these differences are exacerbated as the complexity of the economics-related prompts increased. These findings are relevant to both students and educators; students can choose the most appropriate chatbots to better understand economics concepts and thought processes, while educators can design their instruction and assessment while recognizing the support and resources students have access to through AI chatbot platforms.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Economics and Business, Kalamazoo College, Kalamazoo, Michigan, United States of America.
- Department of Academic Development, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates.
- School of Engineering, Applied Science and Technology, Canadian University Dubai, Dubai, United Arab Emirates.
- College of Interdisciplinary Studies, Zayed University, Dubai, United Arab Emirates.
ORCID-Profile: Patrik T Hultberg, Linda Smail
Zitieren
APA
Hultberg, P. T., Santandreu Calonge, D., Kamalov, F., & Smail, L. (2024). Comparing and assessing four AI chatbots’ competence in economics. PLOS ONE. https://doi.org/10.1371/journal.pone.0297804
Vancouver
Hultberg PT, Santandreu Calonge D, Kamalov F, Smail L. Comparing and assessing four AI chatbots’ competence in economics. PLOS ONE. 2024. doi:10.1371/journal.pone.0297804
BibTeX
@article{Hultberg2024,
author = {Hultberg, Patrik T and Santandreu Calonge, David and Kamalov, Firuz and Smail, Linda},
title = {Comparing and assessing four AI chatbots’ competence in economics},
journal = {PLOS ONE},
year = {2024},
doi = {10.1371/journal.pone.0297804}
}
EUVIMED