EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

An analysis of Watson vs. BARD vs. ChatGPT: The Jeopardy! Challenge

Daniel E. O'Leary

AI Magazine · 2023-09 · 44 (3) · 282-295

DOI: 10.1002/aaai.12118 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AI Magazine, online erschienen am 30.08.2023. Verfasst von Daniel E. O'Leary. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The recently released BARD and ChatGPT have generated substantial interest from a range of researchers and institutions concerned about the impact on education, medicine, law and more. This paper uses questions from the Watson Jeopardy! Challenge to compare BARD, ChatGPT, and Watson. Using those, Jeopardy! questions, we find that for high confidence Watson questions the three systems perform with similar accuracy as Watson. We also find that both BARD and ChatGPT perform with the accuracy of a human expert and that the sets of their correct answers are rated highly similar using a Tanimoto similarity score. However, in addition, we find that both systems can change their solutions to the same input information on subsequent uses. When given the same Jeopardy! category and question multiple times, both BARD and ChatGPT can generate different and conflicting answers. As a result, the paper examines the characteristics of some of those questions that generate different answers to the same inputs. Finally, the paper discusses some of the implications of finding the different answers and the impact of the lack of reproducibility on testing such systems.

Zitieren

APA

O'Leary, D. E. (2023). An analysis of Watson vs. BARD vs. ChatGPT: The Jeopardy! Challenge. AI Magazine, 44(3), 282-295. https://doi.org/10.1002/aaai.12118

Vancouver

O'Leary DE. An analysis of Watson vs. BARD vs. ChatGPT: The Jeopardy! Challenge. AI Magazine. 2023;44(3):282-295. doi:10.1002/aaai.12118

BibTeX
@article{OLeary2023,
  author = {O'Leary, Daniel E.},
  title = {An analysis of Watson vs. BARD vs. ChatGPT: The Jeopardy! Challenge},
  journal = {AI Magazine},
  year = {2023},
  volume = {44},
  number = {3},
  pages = {282--295},
  doi = {10.1002/aaai.12118}
}
Kostenlos starten →