A method for rapid machine learning development for data mining with doctor-in-the-loop
PLOS ONE · 2023
DOI: 10.1371/journal.pone.0284965 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, 2023. Verfasst von Neva J Bull und 3 weiteren Autor:innen. Beteiligte Einrichtung: School of Psychological Sciences, University of Newcastle, Callaghan, NSW, Australia. Hauptthemen laut MEDLINE: Brain Ischemia, Stroke. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 15 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Classifying free-text from historical databases into research-compatible formats is a barrier for clinicians undertaking audit and research projects. The aim of this study was to (a) develop interactive active machine-learning model training methodology using readily available software that was (b) easily adaptable to a wide range of natural language databases and allowed customised researcher-defined categories, and then (c) evaluate the accuracy and speed of this model for classifying free text from two unique and unrelated clinical notes into coded data. A user interface for medical experts to train and evaluate the algorithm was created. Data requiring coding in the form of two independent databases of free-text clinical notes, each of unique natural language structure. Medical experts defined categories relevant to research projects and performed ‘label-train-evaluate’ loops on the training data set. A separate dataset was used for validation, with the medical experts blinded to the label given by the algorithm. The first dataset was 32,034 death certificate records from Northern Territory Births Deaths and Marriages, which were coded into 3 categories: haemorrhagic stroke, ischaemic stroke or no stroke. The second dataset was 12,039 recorded episodes of aeromedical retrieval from two prehospital and retrieval services in Northern Territory, Australia, which were coded into 5 categories: medical, surgical, trauma, obstetric or psychiatric. For the first dataset, macro-accuracy of the algorithm was 94.7%. For the second dataset, macro-accuracy was 92.4%. The time taken to develop and train the algorithm was 124 minutes for the death certificate coding, and 144 minutes for the aeromedical retrieval coding. This machine-learning training method was able to classify free-text clinical notes quickly and accurately from two different health datasets into categories of relevance to clinicians undertaking health service research.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- School of Psychological Sciences, University of Newcastle, Callaghan, NSW, Australia.
- Hunter Medical Research Institute, John Hunter Hospital, New Lambton Heights, NSW, Australia.
- Alice Springs Hospital, Alice Springs, NT, Australia.
- School of Biomedical Sciences and Pharmacy, University of Newcastle, Callaghan, NSW, Australia.
- Department of Neurology, John Hunter Hospital, New Lambton Heights, NSW, Australia.
- National Centre of Epidemiology and Population Health, Australian National University, Canberra, ACT, Australia.
ORCID-Profile: Neva J Bull, Neil J Spratt
Zitieren
APA
Bull, N. J., Honan, B., Spratt, N. J., & Quilty, S. (2023). A method for rapid machine learning development for data mining with doctor-in-the-loop. PLOS ONE. https://doi.org/10.1371/journal.pone.0284965
Vancouver
Bull NJ, Honan B, Spratt NJ, Quilty S. A method for rapid machine learning development for data mining with doctor-in-the-loop. PLOS ONE. 2023. doi:10.1371/journal.pone.0284965
BibTeX
@article{Bull2023,
author = {Bull, Neva J and Honan, Bridget and Spratt, Neil J and Quilty, Simon},
title = {A method for rapid machine learning development for data mining with doctor-in-the-loop},
journal = {PLOS ONE},
year = {2023},
doi = {10.1371/journal.pone.0284965}
}
EUVIMED