Validity Evidence for an Open-Access Internal Medicine Point-of-care Ultrasound Image Interpretation Assessment

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Background: Point-of-care ultrasound (POCUS) has expanding utility in internal medicine practice, but there is a paucity of standardized multiple-choice question (MCQ) POCUS assessments with validity evidence for internal medicine residents. We aimed to develop a POCUS assessment that tests major internal medicine concepts and then evaluate its validity. Methods: We used published internal medicine consensus recommendations to determine the main concepts to evaluate in our assessment. Three investigators drafted multiple-choice questions for the assessment, which were then distributed to 5 subject-matter experts (SMEs) across 4 institutions, who iteratively evaluated and provided recommendations on content validity until predefined content validity indices were achieved. We distributed these questions to internal medicine physicians trained and untrained in POCUS, and then calculated quality metrics. Results: We distributed 59 MCQs to SMEs, and then narrowed them to meet predefined content validity index cutoffs for reliability (r-CVI), usefulness (u-CVI), and clarity (c-CVI). The 36 final questions had high average content ratings (r-CVI: 0.92; u-CVI: 0.92; c-CVI: 0.89). The reliability of these questions was demonstrated by a KR-20 of 0.81. Evidence for internal structure was shown with an average discrimination index of 0.47. Relationship to other variables was demonstrated by a significantly better average performance by 5 trained versus 16 untrained assessment-takers (trained-84%, untrained-60%, p