Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?
PLOS ONE · 2024
DOI: 10.1371/journal.pone.0297526 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Duncan A Q Moore und 2 weiteren Autor:innen. Beteiligte Einrichtung: SPRU (Science Policy Research Unit), University of Sussex, Brighton, United Kingdom. Hauptthemen laut MEDLINE: Abstracting and Indexing, Medical Subject Headings. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Research Council.
Abstract (Originalsprache)
The Medical Subject Headings (MeSH) thesaurus is a controlled vocabulary developed by the U.S. National Library of Medicine (NLM) for classifying journal articles. It is increasingly used by researchers studying medical innovation to classify text into disease areas and other categories. Although this process was once manual, human indexers are now assisted by algorithms that automate some of the indexing process. NLM has made one of their algorithms, the Medical Text Indexer (MTI), available to researchers. MTI can be used to easily assign MeSH descriptors to arbitrary text, including from document types other than publications. However, the reliability of extending MTI to other document types has not been studied directly. To assess this, we collected text from grants, patents, and drug indications, and compared MTI’s classification to expert manual classification of the same documents. We examined MTI’s recall (how often correct terms were identified) and found that MTI identified 78% of expert-classified MeSH descriptors for grants, 78% for patents, and 86% for drug indications. This high recall could be driven merely by excess suggestions (at an extreme, all diseases being assigned to a piece of text); therefore, we also examined precision (how often identified terms were correct) and found that most MTI outputs were also identified by expert manual classification: precision was 53% for grant text, 73% for patent text, and 64% for drug indications. Additionally, we found that recall and precision could be improved by (i) utilizing ranking scores provided by MTI, (ii) excluding long documents, and (iii) aggregating to higher MeSH categories. For simply detecting the presence of any disease, MTI showed > 94% recall and > 87% precision. Our overall assessment is that MTI is a potentially useful tool for researchers wishing to classify texts from a variety of sources into disease areas.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- SPRU (Science Policy Research Unit), University of Sussex, Brighton, United Kingdom.
- School of Public Affairs, Arizona State University, Tempe, Arizona, United States of America.
ORCID-Profile: Duncan A Q Moore, Ohid Yaqub
Förderung
- European Research Council
Zitieren
APA
Moore, D. A. Q., Yaqub, O., & Sampat, B. N. (2024). Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?. PLOS ONE. https://doi.org/10.1371/journal.pone.0297526
Vancouver
Moore DAQ, Yaqub O, Sampat BN. Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?. PLOS ONE. 2024. doi:10.1371/journal.pone.0297526
BibTeX
@article{Moore2024,
author = {Moore, Duncan A Q and Yaqub, Ohid and Sampat, Bhaven N},
title = {Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?},
journal = {PLOS ONE},
year = {2024},
doi = {10.1371/journal.pone.0297526}
}
EUVIMED