EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Do Large Language Models Show Human-like Biases? Exploring Confidence—Competence Gap in AI

Aniket Kumar Singh, Bishal Lamichhane, Suman Devkota, Uttam Dhakal, Chandra Dhakal

Information · 06.02.2024 · 15 (2) · 92

DOI: 10.3390/info15020092 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information, online erschienen am 06.02.2024. Verfasst von Aniket Kumar Singh und 4 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study investigates self-assessment tendencies in Large Language Models (LLMs), examining if patterns resemble human cognitive biases like the Dunning–Kruger effect. LLMs, including GPT, BARD, Claude, and LLaMA, are evaluated using confidence scores on reasoning tasks. The models provide self-assessed confidence levels before and after responding to different questions. The results show cases where high confidence does not correlate with correctness, suggesting overconfidence. Conversely, low confidence despite accurate responses indicates potential underestimation. The confidence scores vary across problem categories and difficulties, reducing confidence for complex queries. GPT-4 displays consistent confidence, while LLaMA and Claude demonstrate more variations. Some of these patterns resemble the Dunning–Kruger effect, where incompetence leads to inflated self-evaluations. While not conclusively evident, these observations parallel this phenomenon and provide a foundation to further explore the alignment of competence and confidence in LLMs. As LLMs continue to expand their societal roles, further research into their self-assessment mechanisms is warranted to fully understand their capabilities and limitations.

Zitieren

APA

Singh, A. K., Lamichhane, B., Devkota, S., Dhakal, U., & Dhakal, C. (2024). Do Large Language Models Show Human-like Biases? Exploring Confidence—Competence Gap in AI. Information, 15(2), 92. https://doi.org/10.3390/info15020092

Vancouver

Singh AK, Lamichhane B, Devkota S, Dhakal U, Dhakal C. Do Large Language Models Show Human-like Biases? Exploring Confidence—Competence Gap in AI. Information. 2024;15(2):92. doi:10.3390/info15020092

BibTeX
@article{Singh2024,
  author = {Singh, Aniket Kumar and Lamichhane, Bishal and Devkota, Suman and Dhakal, Uttam and Dhakal, Chandra},
  title = {Do Large Language Models Show Human-like Biases? Exploring Confidence—Competence Gap in AI},
  journal = {Information},
  year = {2024},
  volume = {15},
  number = {2},
  pages = {92},
  doi = {10.3390/info15020092}
}
Kostenlos starten →