EUVIMEDScopri EUVIMED
Scoping Review

Data Availability for Critical Appraisal Tools for Evaluating Artificial Intelligence in Clinical Studies: Dataset for a Scoping Review

Juan Bautista Cabello López, Vicente Ruiz García, Miguel Torralba, Miguel Maldonado Fernandez, Marimar Úbeda-Carrillo, Eukene Ansuategi, Luis Ramos-Ruperto, José Ignacio Emparanza, Iratxe Urreta-Barallobre, María-Teresa Iglesias Gaspar, José Ignacio Pijoan Zubizarreta, Amanda J Burls

JMIR Data · 2026-08-28 · 7 · e85688-e85688

DOI: 10.2196/85688 ↗

At a glance

Scoping Review in JMIR Data, published online on 2026-08-28. Written by Juan Bautista Cabello López and 11 co-authors. 23 references.

AbstractDati bibliograficiCiteRelated studies

Abstract (original language)

Background

AI is increasingly used in clinical research, generating a growing need for robust critical appraisal tools to evaluate methodological quality, reporting standards, and potential biases. While traditional instruments exist for conventional clinical studies, specific tools designed for AI-based research are still emerging.

Objective

This dataset accompanies a scoping review that aimed to identify and describe existing critical appraisal tools, reporting frameworks, and bias classification systems applicable to clinical studies using AI, including chatbot-based interventions.

Methods

We systematically searched MEDLINE, Embase, CINAHL, PsycINFO, and IEEE Xplore from inception to April 2024. Eligible studies included those proposing or using tools for critical appraisal, reporting, quality assessment, or risk of bias in AI-related clinical research. Screening and extraction followed JBI and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) recommendations. Data extraction combined human review with a supervised GPT-4o–based retrieval-augmented generation (RAG) process to enhance transparency and reproducibility. All AI-assisted outputs were verified independently by two reviewers.

Results

Seventy records were included: 46 reporting guidelines (comprising 26 guides for reporting AI studies, 16 critical appraisal tools, 2 quality assessment instruments, and 2 risk-of-bias tools), 9 bias classification or bias mitigation studies, and 15 chatbot evaluation studies. All datasets, extraction templates, and RAG prompts are publicly available to facilitate validation and reuse.

Conclusions

This dataset provides a comprehensive overview of critical appraisal and reporting tools for AI-based clinical research. It may support the development of standardized evaluation frameworks and promote transparency in future AI-assisted health studies.

Cite

APA

López, J. B. C., García, V. R., Torralba, M., Fernandez, M. M., Úbeda-Carrillo, M., Ansuategi, E., Ramos-Ruperto, L., Emparanza, J. I., Urreta-Barallobre, I., Gaspar, M. I., Zubizarreta, J. I. P., & Burls, A. J. (2026). Data Availability for Critical Appraisal Tools for Evaluating Artificial Intelligence in Clinical Studies: Dataset for a Scoping Review. JMIR Data, 7, e85688-e85688. https://doi.org/10.2196/85688

Vancouver

López JBC, García VR, Torralba M, Fernandez MM, Úbeda-Carrillo M, Ansuategi E, et al. Data Availability for Critical Appraisal Tools for Evaluating Artificial Intelligence in Clinical Studies: Dataset for a Scoping Review. JMIR Data. 2026;7:e85688-e85688. doi:10.2196/85688

BibTeX
@article{Lpez2026,
  author = {López, Juan Bautista Cabello and García, Vicente Ruiz and Torralba, Miguel and Fernandez, Miguel Maldonado and Úbeda-Carrillo, Marimar and Ansuategi, Eukene and Ramos-Ruperto, Luis and Emparanza, José Ignacio and Urreta-Barallobre, Iratxe and Gaspar, María-Teresa Iglesias and Zubizarreta, José Ignacio Pijoan and Burls, Amanda J},
  title = {Data Availability for Critical Appraisal Tools for Evaluating Artificial Intelligence in Clinical Studies: Dataset for a Scoping Review},
  journal = {JMIR Data},
  year = {2026},
  volume = {7},
  pages = {e85688--e85688},
  doi = {10.2196/85688}
}
Start for free →