EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Towards a Benchmark for Scientific Understanding in Humans and Machines

Kristian Gonzalez Barman, Sascha Caron, Tom Claassen, Henk de Regt

Minds and Machines · 25.04.2024 · 34 (1)

DOI: 10.1007/s11023-024-09657-1 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Minds and Machines, online erschienen am 25.04.2024. Verfasst von Kristian Gonzalez Barman und 3 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Interdisciplinary Research Platform.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Scientific understanding is a fundamental goal of science. However, there is currently no good way to measure the scientific understanding of agents, whether these be humans or Artificial Intelligence systems. Without a clear benchmark, it is challenging to evaluate and compare different levels of scientific understanding. In this paper, we propose a framework to create a benchmark for scientific understanding, utilizing tools from philosophy of science. We adopt a behavioral conception of understanding, according to which genuine understanding should be recognized as an ability to perform certain tasks. We extend this notion of scientific understanding by considering a set of questions that gauge different levels of scientific understanding, covering information retrieval, the capability to arrange information to produce an explanation, and the ability to infer how things would be different under different circumstances. We suggest building a Scientific Understanding Benchmark (SUB), formed by a set of these tests, allowing for the evaluation and comparison of scientific understanding. Benchmarking plays a crucial role in establishing trust, ensuring quality control, and providing a basis for performance evaluation. By aligning machine and human scientific understanding we can improve their utility, ultimately advancing scientific understanding and helping to discover new insights within machines.

Förderung

  • Interdisciplinary Research Platform

Zitieren

APA

Barman, K. G., Caron, S., Claassen, T., & Regt, H. D. (2024). Towards a Benchmark for Scientific Understanding in Humans and Machines. Minds and Machines, 34(1). https://doi.org/10.1007/s11023-024-09657-1

Vancouver

Barman KG, Caron S, Claassen T, Regt HD. Towards a Benchmark for Scientific Understanding in Humans and Machines. Minds and Machines. 2024;34(1). doi:10.1007/s11023-024-09657-1

BibTeX
@article{Barman2024,
  author = {Barman, Kristian Gonzalez and Caron, Sascha and Claassen, Tom and Regt, Henk de},
  title = {Towards a Benchmark for Scientific Understanding in Humans and Machines},
  journal = {Minds and Machines},
  year = {2024},
  volume = {34},
  number = {1},
  doi = {10.1007/s11023-024-09657-1}
}
Kostenlos starten →