Towards a Benchmark for Scientific Understanding in Humans and Machines
Minds and Machines · 25.04.2024 · 34 (1)
DOI: 10.1007/s11023-024-09657-1 ↗Kurz gefasst
Fachartikel in der Zeitschrift Minds and Machines, online erschienen am 25.04.2024. Verfasst von Kristian Gonzalez Barman und 3 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Interdisciplinary Research Platform.
Abstract (Originalsprache)
Scientific understanding is a fundamental goal of science. However, there is currently no good way to measure the scientific understanding of agents, whether these be humans or Artificial Intelligence systems. Without a clear benchmark, it is challenging to evaluate and compare different levels of scientific understanding. In this paper, we propose a framework to create a benchmark for scientific understanding, utilizing tools from philosophy of science. We adopt a behavioral conception of understanding, according to which genuine understanding should be recognized as an ability to perform certain tasks. We extend this notion of scientific understanding by considering a set of questions that gauge different levels of scientific understanding, covering information retrieval, the capability to arrange information to produce an explanation, and the ability to infer how things would be different under different circumstances. We suggest building a Scientific Understanding Benchmark (SUB), formed by a set of these tests, allowing for the evaluation and comparison of scientific understanding. Benchmarking plays a crucial role in establishing trust, ensuring quality control, and providing a basis for performance evaluation. By aligning machine and human scientific understanding we can improve their utility, ultimately advancing scientific understanding and helping to discover new insights within machines.
Förderung
- Interdisciplinary Research Platform
Zitieren
APA
Barman, K. G., Caron, S., Claassen, T., & Regt, H. D. (2024). Towards a Benchmark for Scientific Understanding in Humans and Machines. Minds and Machines, 34(1). https://doi.org/10.1007/s11023-024-09657-1
Vancouver
Barman KG, Caron S, Claassen T, Regt HD. Towards a Benchmark for Scientific Understanding in Humans and Machines. Minds and Machines. 2024;34(1). doi:10.1007/s11023-024-09657-1
BibTeX
@article{Barman2024,
author = {Barman, Kristian Gonzalez and Caron, Sascha and Claassen, Tom and Regt, Henk de},
title = {Towards a Benchmark for Scientific Understanding in Humans and Machines},
journal = {Minds and Machines},
year = {2024},
volume = {34},
number = {1},
doi = {10.1007/s11023-024-09657-1}
}
EUVIMED