Improving the measurement of knowledge in multiple-choice tests: A hierarchical multinomial processing tree approach
Acta Psychologica · 2026
DOI: 10.1016/j.actpsy.2026.107587 ↗Kurz gefasst
Fachartikel in der Zeitschrift Acta Psychologica, 2026. Verfasst von Timo Seitz und 2 weiteren Autor:innen. Beteiligte Einrichtung: University of Mannheim, Mannheim, Germany. Electronic address: timo.seitz@uni-mannheim.de. Hauptthemen laut MEDLINE: Psychometrics, Choice Behavior, Knowledge, Educational Measurement. Die Arbeit nennt 103 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deutsche Forschungsgemeinschaft.
Abstract (Originalsprache)
In multiple-choice (MC) tests, correct item responses can stem from actual mastery as well as from lucky guessing. Several approaches have been developed to address this problem, including the implementation of a "Don't know" option. However, scoring such MC tests is challenging, as traditional scoring techniques are differentially biased against certain response strategies. In this article, we propose a hierarchical multinomial processing tree (MPT) model that allows measuring the latent ability while taking into account the tendency to engage in guessing and the preference for a specific response option when guessing. The proposed MPT model includes crossed-random person and item effects and can be estimated in a Bayesian framework. We applied the model to an empirical dataset of a general knowledge test with a true/false response format and "Don't know" option. The model showed good fit to the data and revealed considerable variability in model parameters across both persons and items. Moreover, we found evidence for an improved measurement of knowledge in the form of higher convergent validities compared to conventional scoring techniques. These results highlight the usefulness of applying hierarchical MPT modeling to psychometric questions. We end the article with a discussion of differences to related models, implications for measurement practice, as well as limitations and future model developments.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- University of Mannheim, Mannheim, Germany. Electronic address: timo.seitz@uni-mannheim.de.
- University of Mannheim, Mannheim, Germany.
Förderung
- Deutsche Forschungsgemeinschaft
Zitieren
APA
Seitz, T., Leipold, F. M., & Protte, J. (2026). Improving the measurement of knowledge in multiple-choice tests: A hierarchical multinomial processing tree approach. Acta Psychologica. https://doi.org/10.1016/j.actpsy.2026.107587
Vancouver
Seitz T, Leipold FM, Protte J. Improving the measurement of knowledge in multiple-choice tests: A hierarchical multinomial processing tree approach. Acta Psychologica. 2026. doi:10.1016/j.actpsy.2026.107587
BibTeX
@article{Seitz2026,
author = {Seitz, Timo and Leipold, Franziska M and Protte, Julius},
title = {Improving the measurement of knowledge in multiple-choice tests: A hierarchical multinomial processing tree approach},
journal = {Acta Psychologica},
year = {2026},
doi = {10.1016/j.actpsy.2026.107587}
}
EUVIMED