EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Improving the measurement of knowledge in multiple-choice tests: A hierarchical multinomial processing tree approach

Timo Seitz, Franziska M Leipold, Julius Protte

Acta Psychologica · 2026

DOI: 10.1016/j.actpsy.2026.107587 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Acta Psychologica, 2026. Verfasst von Timo Seitz und 2 weiteren Autor:innen. Beteiligte Einrichtung: University of Mannheim, Mannheim, Germany. Electronic address: timo.seitz@uni-mannheim.de. Hauptthemen laut MEDLINE: Psychometrics, Choice Behavior, Knowledge, Educational Measurement. Die Arbeit nennt 103 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deutsche Forschungsgemeinschaft.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In multiple-choice (MC) tests, correct item responses can stem from actual mastery as well as from lucky guessing. Several approaches have been developed to address this problem, including the implementation of a "Don't know" option. However, scoring such MC tests is challenging, as traditional scoring techniques are differentially biased against certain response strategies. In this article, we propose a hierarchical multinomial processing tree (MPT) model that allows measuring the latent ability while taking into account the tendency to engage in guessing and the preference for a specific response option when guessing. The proposed MPT model includes crossed-random person and item effects and can be estimated in a Bayesian framework. We applied the model to an empirical dataset of a general knowledge test with a true/false response format and "Don't know" option. The model showed good fit to the data and revealed considerable variability in model parameters across both persons and items. Moreover, we found evidence for an improved measurement of knowledge in the form of higher convergent validities compared to conventional scoring techniques. These results highlight the usefulness of applying hierarchical MPT modeling to psychometric questions. We end the article with a discussion of differences to related models, implications for measurement practice, as well as limitations and future model developments.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PsychometricsChoice BehaviorKnowledgeEducational MeasurementModels, StatisticalHumansBayes TheoremModels, Psychological

Stichwörter der Autor:innen

Bayesian hierarchical modelingCognitive psychometricsGuessingMultinomial processing tree modelsResponse bias

Autor:innen und Institutionen

  • University of Mannheim, Mannheim, Germany. Electronic address: timo.seitz@uni-mannheim.de.
  • University of Mannheim, Mannheim, Germany.

Förderung

  • Deutsche Forschungsgemeinschaft

Zitieren

APA

Seitz, T., Leipold, F. M., & Protte, J. (2026). Improving the measurement of knowledge in multiple-choice tests: A hierarchical multinomial processing tree approach. Acta Psychologica. https://doi.org/10.1016/j.actpsy.2026.107587

Vancouver

Seitz T, Leipold FM, Protte J. Improving the measurement of knowledge in multiple-choice tests: A hierarchical multinomial processing tree approach. Acta Psychologica. 2026. doi:10.1016/j.actpsy.2026.107587

BibTeX
@article{Seitz2026,
  author = {Seitz, Timo and Leipold, Franziska M and Protte, Julius},
  title = {Improving the measurement of knowledge in multiple-choice tests: A hierarchical multinomial processing tree approach},
  journal = {Acta Psychologica},
  year = {2026},
  doi = {10.1016/j.actpsy.2026.107587}
}
Kostenlos starten →