EUVIMEDEUVIMED entdecken
Systematische ÜbersichtOpen Access

Interobserver variability studies in diagnostic imaging: a methodological systematic review

Laura Quinn iD, Konstantinos Tryposkiadis, Jon Deeks, Henrica C W De Vet, Sue Mallett, Lidwine B Mokkink, Yemisi Takwoingi, Sian Taylor-Phillips, Alice Sitch

The British Journal of Radiology · 2023-08 · 96 (1148)

DOI: 10.1259/bjr.20220972 ↗

Kurz gefasst

Systematische Übersicht in der Zeitschrift The British Journal of Radiology, 2023-08. Verfasst von Laura Quinn und 8 weiteren Autor:innen. Beteiligte Einrichtung: Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, United Kingdom. Hauptthemen laut MEDLINE: Research Design, Diagnostic Imaging. Die Arbeit wurde bisher 74-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

To review the methodology of interobserver variability studies; including current practice and quality of conducting and reporting studies.

Methods

Interobserver variability studies between January 2019 and January 2020 were included; extracted data comprised of study characteristics, populations, variability measures, key results, and conclusions. Risk of bias was assessed using the COSMIN tool for assessing reliability and measurement error.

Results

Seventy-nine full-text studies were included covering various imaging tests and clinical areas. The median number of patients was 47 (IQR:23–88), and observers were 4 (IQR:2–7), with sample size justified in 12 (15%) studies. Most studies used static images (n = 75, 95%), where all observers interpreted images for all patients (n = 67, 85%). Intraclass correlation coefficients (ICC) (n = 41, 52%), Kappa (κ) statistics (n = 31, 39%) and percentage agreement (n = 15, 19%) were most commonly used. Interpretation of variability estimates often did not correspond with study conclusions. The COSMIN risk of bias tool gave a very good/adequate rating for 52 studies (66%) including any studies that used variability measures listed in the tool. For studies using static images, some study design standards were not applicable and did not contribute to the overall rating.

Conclusions

Interobserver variability studies have diverse study designs and methods, the impact of which requires further evaluation. Sample size for patients and observers was often small without justification. Most studies report ICC and κ values, which did not always coincide with the study conclusion. High ratings were assigned to many studies using the COSMIN risk of bias tool, with certain standards scored ‘not applicable’ when static images were used. Advances in knowledge: The sample size for both patients and observers was often small without justification. For most studies, observers interpreted static images and did not evaluate the process of acquiring the imaging test, meaning it was not possible to assess many COSMIN risk of bias standards for studies with this design. Most studies reported intraclass correlation coefficient and κ statistics; study conclusions often did not correspond with results.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Research DesignDiagnostic ImagingHumansObserver VariationReproducibility of Results

Autor:innen und Institutionen

  • Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, United Kingdom.
  • NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, United Kingdom.
  • Department of Epidemiology and Data Science, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.
  • Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.
  • Centre for Medical Imaging, University College London, London, United Kingdom.
  • Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, United Kingdom.

ORCID-Profile: Laura Quinn

Zitieren

APA

Quinn, L., Tryposkiadis, K., Deeks, J., De Vet, H. C. W., Mallett, S., Mokkink, L. B., Takwoingi, Y., Taylor-Phillips, S., & Sitch, A. (2023). Interobserver variability studies in diagnostic imaging: a methodological systematic review. The British Journal of Radiology, 96(1148). https://doi.org/10.1259/bjr.20220972

Vancouver

Quinn L, Tryposkiadis K, Deeks J, De Vet HCW, Mallett S, Mokkink LB, et al. Interobserver variability studies in diagnostic imaging: a methodological systematic review. The British Journal of Radiology. 2023;96(1148). doi:10.1259/bjr.20220972

BibTeX
@article{Quinn2023,
  author = {Quinn, Laura and Tryposkiadis, Konstantinos and Deeks, Jon and De Vet, Henrica C W and Mallett, Sue and Mokkink, Lidwine B and Takwoingi, Yemisi and Taylor-Phillips, Sian and Sitch, Alice},
  title = {Interobserver variability studies in diagnostic imaging: a methodological systematic review},
  journal = {The British Journal of Radiology},
  year = {2023},
  volume = {96},
  number = {1148},
  doi = {10.1259/bjr.20220972}
}
Kostenlos starten →