EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?

Duncan A Q Moore iD, Ohid Yaqub iD, Bhaven N Sampat

PLOS ONE · 2024

DOI: 10.1371/journal.pone.0297526 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Duncan A Q Moore und 2 weiteren Autor:innen. Beteiligte Einrichtung: SPRU (Science Policy Research Unit), University of Sussex, Brighton, United Kingdom. Hauptthemen laut MEDLINE: Abstracting and Indexing, Medical Subject Headings. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The Medical Subject Headings (MeSH) thesaurus is a controlled vocabulary developed by the U.S. National Library of Medicine (NLM) for classifying journal articles. It is increasingly used by researchers studying medical innovation to classify text into disease areas and other categories. Although this process was once manual, human indexers are now assisted by algorithms that automate some of the indexing process. NLM has made one of their algorithms, the Medical Text Indexer (MTI), available to researchers. MTI can be used to easily assign MeSH descriptors to arbitrary text, including from document types other than publications. However, the reliability of extending MTI to other document types has not been studied directly. To assess this, we collected text from grants, patents, and drug indications, and compared MTI’s classification to expert manual classification of the same documents. We examined MTI’s recall (how often correct terms were identified) and found that MTI identified 78% of expert-classified MeSH descriptors for grants, 78% for patents, and 86% for drug indications. This high recall could be driven merely by excess suggestions (at an extreme, all diseases being assigned to a piece of text); therefore, we also examined precision (how often identified terms were correct) and found that most MTI outputs were also identified by expert manual classification: precision was 53% for grant text, 73% for patent text, and 64% for drug indications. Additionally, we found that recall and precision could be improved by (i) utilizing ranking scores provided by MTI, (ii) excluding long documents, and (iii) aggregating to higher MeSH categories. For simply detecting the presence of any disease, MTI showed > 94% recall and > 87% precision. Our overall assessment is that MTI is a potentially useful tool for researchers wishing to classify texts from a variety of sources into disease areas.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Abstracting and IndexingMedical Subject HeadingsUnited StatesHumansReproducibility of ResultsAlgorithmsNational Library of Medicine (U.S.)

Autor:innen und Institutionen

  • SPRU (Science Policy Research Unit), University of Sussex, Brighton, United Kingdom.
  • School of Public Affairs, Arizona State University, Tempe, Arizona, United States of America.

ORCID-Profile: Duncan A Q Moore, Ohid Yaqub

Förderung

  • European Research Council

Zitieren

APA

Moore, D. A. Q., Yaqub, O., & Sampat, B. N. (2024). Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?. PLOS ONE. https://doi.org/10.1371/journal.pone.0297526

Vancouver

Moore DAQ, Yaqub O, Sampat BN. Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?. PLOS ONE. 2024. doi:10.1371/journal.pone.0297526

BibTeX
@article{Moore2024,
  author = {Moore, Duncan A Q and Yaqub, Ohid and Sampat, Bhaven N},
  title = {Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?},
  journal = {PLOS ONE},
  year = {2024},
  doi = {10.1371/journal.pone.0297526}
}
Kostenlos starten →