EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Can Machine Learning Be Better than Biased Readers?

Atsuhiro Hibi iD, Rui Zhu iD, Pascal N Tyrrell iD

Tomography · 28.04.2023 · 9 (3) · 901-908

DOI: 10.3390/tomography9030074 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Tomography, online erschienen am 28.04.2023. Verfasst von Atsuhiro Hibi und 2 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Medical Science, University of Toronto, Toronto, ON M5S 1A8, Canada. Hauptthemen laut MEDLINE: Machine Learning, Neural Networks, Computer. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 12 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Nippon Steel Corporation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Training machine learning (ML) models in medical imaging requires large amounts of labeled data. To minimize labeling workload, it is common to divide training data among multiple readers for separate annotation without consensus and then combine the labeled data for training a ML model. This can lead to a biased training dataset and poor ML algorithm prediction performance. The purpose of this study is to determine if ML algorithms can overcome biases caused by multiple readers’ labeling without consensus.

Methods

This study used a publicly available chest X-ray dataset of pediatric pneumonia. As an analogy to a practical dataset without labeling consensus among multiple readers, random and systematic errors were artificially added to the dataset to generate biased data for a binary-class classification task. The Resnet18-based convolutional neural network (CNN) was used as a baseline model. A Resnet18 model with a regularization term added as a loss function was utilized to examine for improvement in the baseline model.

Results

The effects of false positive labels, false negative labels, and random errors (5–25%) resulted in a loss of AUC (0–14%) when training a binary CNN classifier. The model with a regularized loss function improved the AUC (75–84%) over that of the baseline model (65–79%).

Conclusion

This study indicated that it is possible for ML algorithms to overcome individual readers’ biases when consensus is not available. It is recommended to use regularized loss functions when allocating annotation tasks to multiple readers as they are easy to implement and effective in mitigating biased labels.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningNeural Networks, ComputerHumansChildAlgorithmsBias

Stichwörter der Autor:innen

annotation biaschest X-rayconvolutional neural networklabeling consensusmachine learning

Autor:innen und Institutionen

  • Institute of Medical Science, University of Toronto, Toronto, ON M5S 1A8, Canada.
  • Department of Medical Imaging, University of Toronto, Toronto, ON M5T 1W7, Canada.
  • Department of Computer Science, University of Toronto, Toronto, ON M5S 2E4, Canada.
  • Department of Statistical Sciences, University of Toronto, Toronto, ON M5G 1Z5, Canada.

ORCID-Profile: Atsuhiro Hibi, Rui Zhu, Pascal N Tyrrell

Förderung

  • Nippon Steel Corporation

Zitieren

APA

Hibi, A., Zhu, R., & Tyrrell, P. N. (2023). Can Machine Learning Be Better than Biased Readers?. Tomography, 9(3), 901-908. https://doi.org/10.3390/tomography9030074

Vancouver

Hibi A, Zhu R, Tyrrell PN. Can Machine Learning Be Better than Biased Readers?. Tomography. 2023;9(3):901-908. doi:10.3390/tomography9030074

BibTeX
@article{Hibi2023,
  author = {Hibi, Atsuhiro and Zhu, Rui and Tyrrell, Pascal N},
  title = {Can Machine Learning Be Better than Biased Readers?},
  journal = {Tomography},
  year = {2023},
  volume = {9},
  number = {3},
  pages = {901--908},
  doi = {10.3390/tomography9030074}
}
Kostenlos starten →