EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Comparing and assessing four AI chatbots’ competence in economics

Patrik T Hultberg iD, David Santandreu Calonge, Firuz Kamalov, Linda Smail iD

PLOS ONE · 2024

DOI: 10.1371/journal.pone.0297804 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift PLOS ONE, 2024. Verfasst von Patrik T Hultberg und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Economics and Business, Kalamazoo College, Kalamazoo, Michigan, United States of America. Hauptthemen laut MEDLINE: Artificial Intelligence. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Artificial Intelligence (AI) chatbots have emerged as powerful tools in modern academic endeavors, presenting both opportunities and challenges in the learning landscape. They can provide content information and analysis across most academic disciplines, but significant differences exist in terms of response accuracy for conclusions and explanations, as well as word counts. This study explores four distinct AI chatbots, GPT-3.5, GPT-4, Bard, and LLaMA 2, for accuracy of conclusions and quality of explanations in the context of university-level economics. Leveraging Bloom’s taxonomy of cognitive learning complexity as a guiding framework, the study confronts the four AI chatbots with a standard test for university-level understanding of economics, as well as more advanced economics problems. The null hypothesis that all AI chatbots perform equally well on prompts that explore understanding of economics is rejected. The results are that significant differences are observed across the four AI chatbots, and these differences are exacerbated as the complexity of the economics-related prompts increased. These findings are relevant to both students and educators; students can choose the most appropriate chatbots to better understand economics concepts and thought processes, while educators can design their instruction and assessment while recognizing the support and resources students have access to through AI chatbot platforms.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceHumansEconomicsUniversitiesStudentsLearningMaleFemale

Autor:innen und Institutionen

  • Department of Economics and Business, Kalamazoo College, Kalamazoo, Michigan, United States of America.
  • Department of Academic Development, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates.
  • School of Engineering, Applied Science and Technology, Canadian University Dubai, Dubai, United Arab Emirates.
  • College of Interdisciplinary Studies, Zayed University, Dubai, United Arab Emirates.

ORCID-Profile: Patrik T Hultberg, Linda Smail

Zitieren

APA

Hultberg, P. T., Santandreu Calonge, D., Kamalov, F., & Smail, L. (2024). Comparing and assessing four AI chatbots’ competence in economics. PLOS ONE. https://doi.org/10.1371/journal.pone.0297804

Vancouver

Hultberg PT, Santandreu Calonge D, Kamalov F, Smail L. Comparing and assessing four AI chatbots’ competence in economics. PLOS ONE. 2024. doi:10.1371/journal.pone.0297804

BibTeX
@article{Hultberg2024,
  author = {Hultberg, Patrik T and Santandreu Calonge, David and Kamalov, Firuz and Smail, Linda},
  title = {Comparing and assessing four AI chatbots’ competence in economics},
  journal = {PLOS ONE},
  year = {2024},
  doi = {10.1371/journal.pone.0297804}
}
Kostenlos starten →