A Competency Assessment Tool for Liver Contrast-Enhanced Ultrasound: Development and Validation in a Large Multicenter Retrospective Cohort of Trainees (Preprint)
JMIR Medical Education · 2025-10-29
DOI: 10.2196/86730 ↗At a glance
Multizentrische Studie in JMIR Medical Education, published online on 2025-10-29. Written by Yanping Ma and 25 co-authors. Institution: Department of Medical Ultrasonics, Third Affiliated Hospital of Sun Yat-sen University, 600 Tianhe Road, Guangzhou, 510630, China, 86 13925155583. Major MEDLINE topics: Clinical Competence, Liver, Educational Measurement. 1 references. The full text is freely available in PubMed Central.
Abstract (original language)
BackgroundDespite the increasing use of liver contrast-enhanced ultrasound (CEUS), no standardized, organ-specific tool exists for assessing trainees’ cognitive competency. Existing instruments lack multidimensional evaluation and external validation, which limit training quality and patient safety. ObjectiveThis study aims to develop and validate the CEUS Competency Assessment for the Liver (CCAL), a tool to standardize the evaluation of theoretical and cognitive skills in liver CEUS. MethodsThis retrospective cohort study had four stages: (1) item generation from literature and guidelines, (2) 3 Delphi rounds (12/21, 57.1% female experts; mean age 51.0, SD 7.2 y) and cognitive testing (10 trainees) for content validity (>85% agreement) and clarity, (3) internal validation involving 100 sonography trainees (n=66, 66% assistant professors; n=27, 27% associate professors; n=7, 7% professors; n=70, 70% CEUS-naive) with 1 month or more of training at 4 hospitals (2018‐2020), and (4) external validation on 579 sonography trainees (n=299, 51.6% assistant professors; n=183, 31.6% associate professors; n=97, 16.8% professors; n=439, 75.8% CEUS-naive) with 1 or more months of training at 25 multicenter sites (2018‐2020). Primary outcomes were reliability (Cronbach α and Guttman split-half) and validity (exploratory factor analysis and item discrimination), while secondary outcomes included item difficulty distribution, pass rates, and between-group differences. ResultsThe final CCAL comprised 74 items across 3 domains (demographics: 8, self-evaluation: 13, and objective questions: 53). Internal consistency was excellent (Cronbach α=0.916 and Guttman split-half=0.908). Factor analysis confirmed a 3-factor structure explaining 74.6% of the cumulative variance. Item analysis showed a balanced difficulty distribution (35/53, 66.0% easy; 17/53, 32.1% moderate; 1/53, 1.9% difficult items). Scores varied significantly by CEUS training duration (PF2,576F2,576t577dPt577dP ConclusionsThe CCAL is the first liver-specific CEUS competency tool integrating objective knowledge testing with structured self-evaluation. Unlike generic or simulation-based tests, it is organ-specific and validated across a large multicenter cohort, providing a multidimensional framework that identifies specific competency gaps, most notably in advanced image-guided procedures, beyond global pass/fail outcomes. The tool supports competency-based medical education through rigorous data-driven assessment, targeted curricular improvements, cross-institutional benchmarking, and informed training standards. It evaluates cognitive competence exclusively and should not replace procedural assessment in high-stakes decisions. International validation and the establishment of criterion-related and predictive validity are essential next steps.
Subjects
MeSH terms (MEDLINE)
Author keywords
Substances
Authors and institutions
- Department of Medical Ultrasonics, Third Affiliated Hospital of Sun Yat-sen University, 600 Tianhe Road, Guangzhou, 510630, China, 86 13925155583.
- Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
- Department of Health Statistics, School of Preventive Medicine, Air Force Medical University, Xi'an, China.
- Department of Ultrasound, China-Japan Union Hospital of Jilin University, Jilin, China.
- Shanxi Provincial Cancer Hospital, Taiyuan, China.
- Lanzhou University Second Hospital, Lanzhou, China.
Cite
APA
Ma, Y., Zheng, B., Ren, J., He, W., Shang, L., Wang, H., Xue, G., Nie, F., Guo, Y., Luo, Y., Zhang, H., Wang, X., Jing, X., Zhang, R., Yuan, J., Wu, R., Luo, X., Dong, F., Luo, Y., …, & Xie, X. (2025). A Competency Assessment Tool for Liver Contrast-Enhanced Ultrasound: Development and Validation in a Large Multicenter Retrospective Cohort of Trainees (Preprint). JMIR Medical Education. https://doi.org/10.2196/86730
Vancouver
Ma Y, Zheng B, Ren J, He W, Shang L, Wang H, et al. A Competency Assessment Tool for Liver Contrast-Enhanced Ultrasound: Development and Validation in a Large Multicenter Retrospective Cohort of Trainees (Preprint). JMIR Medical Education. 2025. doi:10.2196/86730
BibTeX
@article{Ma2025,
author = {Ma, Yanping and Zheng, Bowen and Ren, Jie and He, Wen and Shang, Lei and Wang, Hui and Xue, Gaiqin and Nie, Fang and Guo, Yanli and Luo, Yukun and Zhang, Hongxia and Wang, Xuan and Jing, Xiang and Zhang, Ruifang and Yuan, Jianjun and Wu, Rong and Luo, Xiaomao and Dong, Fenglin and Luo, Yan and Lu, Qiang and Liao, Jintang and Huang, Pintong and Chen, Qin and Zhu, Jia'an and Xu, Youfeng and Xie, Xiaoyan},
title = {A Competency Assessment Tool for Liver Contrast-Enhanced Ultrasound: Development and Validation in a Large Multicenter Retrospective Cohort of Trainees (Preprint)},
journal = {JMIR Medical Education},
year = {2025},
doi = {10.2196/86730}
}
EUVIMED