Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study
GMS Zeitschrift für Medizinische Ausbildung; 32(4):Doc46 · 15.10.2015 · 32 (4) · Doc46-Doc46
DOI: 10.3205/zma000988 ↗Kurz gefasst
Fachartikel in der Zeitschrift GMS Zeitschrift für Medizinische Ausbildung; 32(4):Doc46, erschienen am 15.10.2015. Verfasst von Stefan Wagener und 20 weiteren Autor:innen. Beteiligte Einrichtung: University of Heidelberg, Faculty of Medicine, Heidelberg, Germany. Hauptthemen laut MEDLINE: Feedback, Students, Medical. Die Arbeit wurde bisher 19-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
Progress tests provide students feedback on their level of proficiency over the course of their medical studies. Peer-assisted learning and competency-based education have become increasingly important in medical education. Although progress tests have been proven to be useful as a longitudinal feedback instrument, there are currently no progress tests that have been created in cooperation with students or that focus on competency in medical education. In this study, we investigated the extent to which students can be included in the development of a progress test and demonstrated that aspects of knowledge related to competency can be represented on a competency-based progress test.
Methods
A two-dimensional blueprint for 144 multiple-choice questions (MCQs) covering groups of medical subjects and groups of competency areas was generated by three expert groups for developing the competency-based progress test. A total of 31 students from seven medical schools in Germany actively participated in this exercise. After completing an intensive and comprehensive training programme, the students generated and reviewed the test questions for the competency-based progress test using a separate platform of the ItemManagementSystem (IMS). This test was administered as a formative test to 469 students in a pilot study in November 2013 at eight medical schools in Germany. The scores were analysed for the overall test and differentiated according to the subject groups and competency areas.
Results
A pool of more than 200 MCQs was compiled by the students for pilot use, of which 118 student-generated MCQs were used in the progress test. University instructors supplemented this pool with 26 MCQs, which primarily addressed the area of scientific skills. The post-review showed that student-generated MCQs were of high quality with regard to test statistic criteria and content. Overall, the progress test displayed a very high reliability. When the academic years were compared, the progress test mapped out over the course of study not only by the overall test but also in terms of the subject groups and competency areas. Outlook: Further development in cooperation with students will be continued. Focus will be on compiling additional questions and test formats that can represent competency at a higher skill level, such as key feature questions, situational judgement test questions and OSCE. In addition, the feedback formats will be successively expanded. The intention is also to offer the formative competency-based progress test online.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- University of Heidelberg, Faculty of Medicine, Heidelberg, Germany.
- University of Heidelberg, Center of Excellence for Assessment in Medicine - Baden-Wuerttemberg, Heidelberg, Germany.
- Albert-Ludwigs-University Freiburg, Freiburg Competence Center for Evaluation of Teaching in Medicine - Baden-Wuerttemberg, Freiburg, Germany.
- University of Tübingen, Faculty of Medicine, Tübingen, Germany.
- University of Ulm, Faculty of Medicine, Ulm, Germany.
- University of Marburg, Faculty of Medicine, Marburg, Germany.
Zitieren
APA
Wagener, S., Möltner, A., Tımbıl, S., Gornostayeva, M., Schultz, J. H., Brüstle, P., Mohr, D., Vander Beken, A., Better, J., Fries, M., Gottschalk, M., Günther, J., Herrmann, L., Kreisel, C., Moczko, T., Illg, C., Jassowicz, A., Müller, A., Niesert, M., …, & Jünger, J. (2015). Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study. GMS Zeitschrift für Medizinische Ausbildung; 32(4):Doc46, 32(4), Doc46-Doc46. https://doi.org/10.3205/zma000988
Vancouver
Wagener S, Möltner A, Tımbıl S, Gornostayeva M, Schultz JH, Brüstle P, et al. Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study. GMS Zeitschrift für Medizinische Ausbildung; 32(4):Doc46. 2015;32(4):Doc46-Doc46. doi:10.3205/zma000988
BibTeX
@article{Wagener2015,
author = {Wagener, Stefan and Möltner, Andreas and Tımbıl, Sevgi and Gornostayeva, Maryna and Schultz, Jobst-Hendrik and Brüstle, Peter and Mohr, Daniela and Vander Beken, Anna and Better, Julian and Fries, Martin and Gottschalk, Marc and Günther, Janine and Herrmann, Laura and Kreisel, Christian and Moczko, Tobias and Illg, Claudius and Jassowicz, Adam and Müller, Andreas and Niesert, Moritz and Strübing, Felix and Jünger, Jana},
title = {Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study},
journal = {GMS Zeitschrift für Medizinische Ausbildung; 32(4):Doc46},
year = {2015},
volume = {32},
number = {4},
pages = {Doc46--Doc46},
doi = {10.3205/zma000988}
}
EUVIMED