EUVIMEDEUVIMED entdecken
Open Access

Automated ICD-10–Anchored Classification of Primary Care Text Data: Development and Evaluation of a Custom Multilabel Classifier

Christina Haag iD, Thomas Grischott iD, Jakob M Burgstaller iD, Stefan Markun iD, Oliver Senn iD, Viktor von Wyl iD

JMIR Medical Informatics · 06.04.2026 · 14 · e86533-e86533

DOI: 10.2196/86533 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Medical Informatics, online erschienen am 06.04.2026. Verfasst von Christina Haag und 5 weiteren Autor:innen. Beteiligte Einrichtung: Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Hirschengraben 84, Zurich, 8001, Switzerland, +41 44 63 46380. Hauptthemen laut MEDLINE: International Classification of Diseases, Electronic Health Records, Primary Health Care. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 17 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Electronic medical records are a vast and valuable source of information, useful for tasks such as estimating disease prevalence. However, in routine primary care, much of this information is in free-text format rather than in a structured form and, therefore, not readily amenable to analysis. Manual coding of this textual data is both time-consuming and resource-intensive, making it impractical for large datasets. Although powerful open-source language models offer new opportunities for automated coding, their use on short heterogeneous primary care notes, particularly in German-language settings, remains insufficiently studied.

Objective

By providing hands-on guidance for applied health researchers, this study aims to demonstrate the effective and accurate automatic classification of free-text notes using a language model fine-tuned for automated International Statistical Classification of Diseases, Tenth Revision ( ICD-10 ) coding.

Methods

Building on the extensive Family Medicine Research Using Electronic Medical Records (FIRE) routine database from the Institute of Primary Care at the University Hospital Zurich and the University of Zurich, we trained a large language model–based multilabel classifier on a dataset of 38,728 free-text notes, which had been manually categorized into 47 classes using specific ICD-10 codes and code ranges or nondiagnostic/ad hoc labels (eg, “unclear diagnosis,” “status post”). We stratified the labeled data into training (70%), validation (15%), and posttraining test (15%) sets, ensuring similar label distributions across these sets. Using the Transformers Python library, we trained the model over 10 epochs and evaluated it on the posttraining test dataset.

Results

Across 48 classes, the FIRE classifier achieved strong performance on the held-out posttraining set, with F 1 -scores of 0.85 (micro, overall across all predictions), 0.86 (macro, mean of per-class scores treating classes equally), and 0.84 (weighted, per-class scores weighted by class frequency).

Conclusions

This study demonstrates steps for training open-source large language models and highlights the potential to streamline and scale the extraction of diagnostic information for practical applications. Our model can be robustly deployed, for example, for prescreening and labeling of free-text information, thus potentially reducing the burden of repetitive and error-prone manual handling.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

International Classification of DiseasesElectronic Health RecordsPrimary Health CareHumansClassification AlgorithmsSwitzerlandLarge Language Models

Stichwörter der Autor:innen

ICD-10 codingautomated codingclinical text miningelectronic medical recordsfree-text classificationhealth informaticslanguage modelsnatural language processingprimary care

Autor:innen und Institutionen

  • Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Hirschengraben 84, Zurich, 8001, Switzerland, +41 44 63 46380.
  • Institute for Implementation Science in Health Care, University of Zurich, Universitätstrasse 84, Zurich, 8006, Switzerland, 41 44 63 46380.
  • Institute of Primary Care, University Hospital Zurich, University of Zurich, Zurich, Switzerland.

ORCID-Profile: Christina Haag, Thomas Grischott, Jakob M Burgstaller, Stefan Markun, Oliver Senn, Viktor von Wyl

Zitieren

APA

Haag, C., Grischott, T., Burgstaller, J. M., Markun, S., Senn, O., & von Wyl, V. (2026). Automated ICD-10–Anchored Classification of Primary Care Text Data: Development and Evaluation of a Custom Multilabel Classifier. JMIR Medical Informatics, 14, e86533-e86533. https://doi.org/10.2196/86533

Vancouver

Haag C, Grischott T, Burgstaller JM, Markun S, Senn O, von Wyl V. Automated ICD-10–Anchored Classification of Primary Care Text Data: Development and Evaluation of a Custom Multilabel Classifier. JMIR Medical Informatics. 2026;14:e86533-e86533. doi:10.2196/86533

BibTeX
@article{Haag2026,
  author = {Haag, Christina and Grischott, Thomas and Burgstaller, Jakob M and Markun, Stefan and Senn, Oliver and von Wyl, Viktor},
  title = {Automated ICD-10–Anchored Classification of Primary Care Text Data: Development and Evaluation of a Custom Multilabel Classifier},
  journal = {JMIR Medical Informatics},
  year = {2026},
  volume = {14},
  pages = {e86533--e86533},
  doi = {10.2196/86533}
}
Kostenlos starten →