EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Robustness of large language models in moral judgements

Soyoung Oh iD, Vera Demberg iD

Royal Society Open Science · 2025-04 · 12 (4)

DOI: 10.1098/rsos.241229 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Royal Society Open Science, online erschienen am 23.04.2025. Verfasst von Soyoung Oh und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, Language Science and Technology, Saarland University, Saarbrücken, Germany. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the advent of large language models (LLMs), there has been a growing interest in analysing the preferences encoded in LLMs in the context of morality. Recent work has tested LLMs on various moral judgement tasks and drawn conclusions regarding the alignment between LLMs and humans. The present contribution critically assesses the validity of the method and results employed in previous work for eliciting moral judgements from LLMs. We find that previous results are confounded by biases in the presentation of the options in moral judgement tasks and that LLM responses are highly sensitive to prompt formulation variants as simple as changing ‘Case 1’ and ‘Case 2’ to ‘(A)’ and ‘(B)’. Our results hence indicate that previous conclusions on moral judgements of LLMs cannot be upheld. We make recommendations for more sound methodological setups for future studies.

Schlagwörter

Stichwörter der Autor:innen

large language modelmoral reasoningrobustness

Autor:innen und Institutionen

  • Department of Computer Science, Language Science and Technology, Saarland University, Saarbrücken, Germany.

ORCID-Profile: Soyoung Oh, Vera Demberg

Förderung

  • European Research Council

Zitieren

APA

Oh, S., & Demberg, V. (2025). Robustness of large language models in moral judgements. Royal Society Open Science, 12(4). https://doi.org/10.1098/rsos.241229

Vancouver

Oh S, Demberg V. Robustness of large language models in moral judgements. Royal Society Open Science. 2025;12(4). doi:10.1098/rsos.241229

BibTeX
@article{Oh2025,
  author = {Oh, Soyoung and Demberg, Vera},
  title = {Robustness of large language models in moral judgements},
  journal = {Royal Society Open Science},
  year = {2025},
  volume = {12},
  number = {4},
  doi = {10.1098/rsos.241229}
}
Kostenlos starten →