Data Availability for Critical Appraisal Tools for Evaluating Artificial Intelligence in Clinical Studies: Dataset for a Scoping Review
JMIR Data · 28.08.2026 · 7 · e85688-e85688
DOI: 10.2196/85688 ↗Kurz gefasst
Scoping Review in der Zeitschrift JMIR Data, online erschienen am 28.08.2026. Verfasst von Juan Bautista Cabello López und 11 weiteren Autor:innen. Die Arbeit nennt 23 Literaturangaben.
Abstract (Originalsprache)
Background
AI is increasingly used in clinical research, generating a growing need for robust critical appraisal tools to evaluate methodological quality, reporting standards, and potential biases. While traditional instruments exist for conventional clinical studies, specific tools designed for AI-based research are still emerging.
Objective
This dataset accompanies a scoping review that aimed to identify and describe existing critical appraisal tools, reporting frameworks, and bias classification systems applicable to clinical studies using AI, including chatbot-based interventions.
Methods
We systematically searched MEDLINE, Embase, CINAHL, PsycINFO, and IEEE Xplore from inception to April 2024. Eligible studies included those proposing or using tools for critical appraisal, reporting, quality assessment, or risk of bias in AI-related clinical research. Screening and extraction followed JBI and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) recommendations. Data extraction combined human review with a supervised GPT-4o–based retrieval-augmented generation (RAG) process to enhance transparency and reproducibility. All AI-assisted outputs were verified independently by two reviewers.
Results
Seventy records were included: 46 reporting guidelines (comprising 26 guides for reporting AI studies, 16 critical appraisal tools, 2 quality assessment instruments, and 2 risk-of-bias tools), 9 bias classification or bias mitigation studies, and 15 chatbot evaluation studies. All datasets, extraction templates, and RAG prompts are publicly available to facilitate validation and reuse.
Conclusions
This dataset provides a comprehensive overview of critical appraisal and reporting tools for AI-based clinical research. It may support the development of standardized evaluation frameworks and promote transparency in future AI-assisted health studies.
Zitieren
APA
López, J. B. C., García, V. R., Torralba, M., Fernandez, M. M., Úbeda-Carrillo, M., Ansuategi, E., Ramos-Ruperto, L., Emparanza, J. I., Urreta-Barallobre, I., Gaspar, M. I., Zubizarreta, J. I. P., & Burls, A. J. (2026). Data Availability for Critical Appraisal Tools for Evaluating Artificial Intelligence in Clinical Studies: Dataset for a Scoping Review. JMIR Data, 7, e85688-e85688. https://doi.org/10.2196/85688
Vancouver
López JBC, García VR, Torralba M, Fernandez MM, Úbeda-Carrillo M, Ansuategi E, et al. Data Availability for Critical Appraisal Tools for Evaluating Artificial Intelligence in Clinical Studies: Dataset for a Scoping Review. JMIR Data. 2026;7:e85688-e85688. doi:10.2196/85688
BibTeX
@article{Lpez2026,
author = {López, Juan Bautista Cabello and García, Vicente Ruiz and Torralba, Miguel and Fernandez, Miguel Maldonado and Úbeda-Carrillo, Marimar and Ansuategi, Eukene and Ramos-Ruperto, Luis and Emparanza, José Ignacio and Urreta-Barallobre, Iratxe and Gaspar, María-Teresa Iglesias and Zubizarreta, José Ignacio Pijoan and Burls, Amanda J},
title = {Data Availability for Critical Appraisal Tools for Evaluating Artificial Intelligence in Clinical Studies: Dataset for a Scoping Review},
journal = {JMIR Data},
year = {2026},
volume = {7},
pages = {e85688--e85688},
doi = {10.2196/85688}
}
EUVIMED