EUVIMEDOpdag EUVIMED
Qualitative StudieStudienprotokollOpen access

Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies Using AI (QUADAS-AI): Protocol for a Qualitative Study

Ahmad Guni iD, Viknesh Sounderajah iD, Penny Whiting iD, Patrick Bossuyt iD, Ara Darzi iD, Hutan Ashrafian iD

JMIR Research Protocols · 2024-09-18 · 13 · e58202

DOI: 10.2196/58202 ↗

At a glance

Qualitative Studie in JMIR Research Protocols, published online on 2024-09-18. Written by Ahmad Guni and 5 co-authors. Institution: Institute of Global Health Innovation, Imperial College London, London, United Kingdom. Major MEDLINE topics: Artificial Intelligence, Qualitative Research. Cited 80 times so far (Crossref); 18 references. The full text is freely available in PubMed Central. Funded by: National Institute for Health and Care Research (NIHR) Imperial Biomedical Research Centre (BRC).

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Quality assessment of diagnostic accuracy studies (QUADAS), and more recently QUADAS-2, were developed to aid the evaluation of methodological quality within primary diagnostic accuracy studies. However, its current form, QUADAS-2 does not address the unique considerations raised by artificial intelligence (AI)–centered diagnostic systems. The rapid progression of the AI diagnostics field mandates suitable quality assessment tools to determine the risk of bias and applicability, and subsequently evaluate translational potential for clinical practice.

Objective

We aim to develop an AI-specific QUADAS (QUADAS-AI) tool that addresses the specific challenges associated with the appraisal of AI diagnostic accuracy studies. This paper describes the processes and methods that will be used to develop QUADAS-AI.

Methods

The development of QUADAS-AI can be distilled into 3 broad stages. Stage 1—a project organization phase had been undertaken, during which a project team and a steering committee were established. The steering committee consists of a panel of international experts representing diverse stakeholder groups. Following this, the scope of the project was finalized. Stage 2—an item generation process will be completed following (1) a mapping review, (2) a meta-research study, (3) a scoping survey of international experts, and (4) a patient and public involvement and engagement exercise. Candidate items will then be put forward to the international Delphi panel to achieve consensus for inclusion in the revised tool. A modified Delphi consensus methodology involving multiple online rounds and a final consensus meeting will be carried out to refine the tool, following which the initial QUADAS-AI tool will be drafted. A piloting phase will be carried out to identify components that are considered to be either ambiguous or missing. Stage 3—once the steering committee has finalized the QUADAS-AI tool, specific dissemination strategies will be aimed toward academic, policy, regulatory, industry, and public stakeholders, respectively.

Results

As of July 2024, the project organization phase, as well as the mapping review and meta-research study, have been completed. We aim to complete the item generation, including the Delphi consensus, and finalize the tool by the end of 2024. Therefore, QUADAS-AI will be able to provide a consensus-derived platform upon which stakeholders may systematically appraise the methodological quality associated with AI diagnostic accuracy studies by the beginning of 2025.

Conclusions

AI-driven systems comprise an increasingly significant proportion of research in clinical diagnostics. Through this process, QUADAS-AI will aid the evaluation of studies in this domain in order to identify bias and applicability concerns. As such, QUADAS-AI may form a key part of clinical, governmental, and regulatory evaluation frameworks for AI diagnostic systems globally. International Registered Report Identifier (IRRID) DERR1-10.2196/58202

Subjects

MeSH terms (MEDLINE)

Artificial IntelligenceQualitative ResearchHumansResearch DesignQuality Assurance, Health CareDelphi Technique

Author keywords

AIAI-drivenAI-specific quality assessment of diagnostic accuracy studiesQUADAS-AIaccuracyartificial intelligenceassessment toolbiasclinical practicediagnosticdiagnostic servicediagnosticsevaluationevidence synthesisquality assessment

Authors and institutions

  • Institute of Global Health Innovation, Imperial College London, London, United Kingdom.
  • Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom.
  • Department of Epidemiology & Data Science, Amsterdam University Medical Centres, Amsterdam, Netherlands.

ORCID profiles: Ahmad Guni, Viknesh Sounderajah, Penny Whiting, Patrick Bossuyt, Ara Darzi, Hutan Ashrafian

Funding

  • National Institute for Health and Care Research (NIHR) Imperial Biomedical Research Centre (BRC)

Cite

APA

Guni, A., Sounderajah, V., Whiting, P., Bossuyt, P., Darzi, A., & Ashrafian, H. (2024). Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies Using AI (QUADAS-AI): Protocol for a Qualitative Study. JMIR Research Protocols, 13, e58202. https://doi.org/10.2196/58202

Vancouver

Guni A, Sounderajah V, Whiting P, Bossuyt P, Darzi A, Ashrafian H. Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies Using AI (QUADAS-AI): Protocol for a Qualitative Study. JMIR Research Protocols. 2024;13:e58202. doi:10.2196/58202

BibTeX
@article{Guni2024,
  author = {Guni, Ahmad and Sounderajah, Viknesh and Whiting, Penny and Bossuyt, Patrick and Darzi, Ara and Ashrafian, Hutan},
  title = {Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies Using AI (QUADAS-AI): Protocol for a Qualitative Study},
  journal = {JMIR Research Protocols},
  year = {2024},
  volume = {13},
  pages = {e58202},
  doi = {10.2196/58202}
}
Start for free →