EUVIMEDEUVIMED entdecken
Crossref

Estimating the Irreducible Uncertainty in Visual Diagnosis: Statistical Modeling of Skill Using Response Models

Martin V Pusic iD, Amy Rapkiewicz, Tenko Raykov, Jonathan Melamed

Medical Decision Making · 2023

DOI: 10.1177/0272989x231162095 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Medical Decision Making, 2023. Verfasst von Martin V Pusic und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department Pediatrics and Emergency Medicine, Harvard Medical School, Boston, MA, USA. Hauptthemen laut MEDLINE: Prostatic Neoplasms. Die Arbeit nennt 24 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

For the representative problem of prostate cancer grading, we sought to simultaneously model both the continuous nature of the case spectrum and the decision thresholds of individual pathologists, allowing quantitative comparison of how they handle cases at the borderline between diagnostic categories.

Methods

Experts and pathology residents each rated a standardized set of prostate cancer histopathological images on the International Society of Urological Pathologists (ISUP) scale used in clinical practice. They diagnosed 50 histologic cases with a range of malignancy, including intermediate cases in which clear distinction was difficult. We report a statistical model showing the degree to which each individual participant can separate the cases along the latent decision spectrum.

Results

The slides were rated by 36 physicians in total: 23 ISUP pathologists and 13 residents. As anticipated, the cases showed a full continuous range of diagnostic severity. Cases ranged along a logit scale consistent with the consensus rating (Consensus ISUP 1: mean −0.93 [95% confidence interval {CI} −1.10 to −0.78], ISUP 2: −0.19 logits [−0.27 to −0.12]; ISUP 3: 0.56 logits [0.06–1.06]; ISUP 4 1.24 logits [1.10–1.38]; ISUP 5: 1.92 [1.80–2.04]). The best raters were able to meaningfully discriminate between all 5 ISUP categories, showing intercategory thresholds that were quantifiably precise and meaningful.

Conclusions

We present a method that allows simultaneous quantification of both the confusability of a particular case and the skill with which raters can distinguish the cases.

Implications

The technique generalizes beyond the current example to other clinical situations in which a diagnostician must impose an ordinal rating on a biological spectrum. Highlights Question: How can we quantify skill in visual diagnosis for cases that sit at the border between 2 ordinal categories—cases that are inherently difficult to diagnose?

Findings

In this analysis of pathologists and residents rating prostate biopsy specimens, decision-aligned response models are calculated that show how pathologists would be likely to classify any given case on the diagnostic spectrum. Decision thresholds are shown to vary in their location and precision. Significance: Improving on traditional measures such as kappa and receiver-operating characteristic curves, this specialization of item response models allows better individual feedback to both trainees and pathologists, including better quantification of acceptable decision variation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Prostatic NeoplasmsMaleHumansNeoplasm GradingUncertaintyModels, StatisticalPathologists

Stichwörter der Autor:innen

assessmentbenchmarkingdiagnostic errorsitem response theorymedical educationpathologyprostatic neoplasmspsychometricsuncertaintyvisual diagnosis

Autor:innen und Institutionen

  • Department Pediatrics and Emergency Medicine, Harvard Medical School, Boston, MA, USA.
  • Department of Pathology, NYU Long Island School of Medicine, New York, NY, USA.
  • College of Education, Michigan State University. East Lansing, MI, USA.

ORCID-Profile: Martin V Pusic

Zitieren

APA

Pusic, M. V., Rapkiewicz, A., Raykov, T., & Melamed, J. (2023). Estimating the Irreducible Uncertainty in Visual Diagnosis: Statistical Modeling of Skill Using Response Models. Medical Decision Making. https://doi.org/10.1177/0272989x231162095

Vancouver

Pusic MV, Rapkiewicz A, Raykov T, Melamed J. Estimating the Irreducible Uncertainty in Visual Diagnosis: Statistical Modeling of Skill Using Response Models. Medical Decision Making. 2023. doi:10.1177/0272989x231162095

BibTeX
@article{Pusic2023,
  author = {Pusic, Martin V and Rapkiewicz, Amy and Raykov, Tenko and Melamed, Jonathan},
  title = {Estimating the Irreducible Uncertainty in Visual Diagnosis: Statistical Modeling of Skill Using Response Models},
  journal = {Medical Decision Making},
  year = {2023},
  doi = {10.1177/0272989x231162095}
}
Kostenlos starten →