EUVIMEDScopri EUVIMED
Open access

Autonomous International Classification of Diseases Coding Using Pretrained Language Models and Advanced Prompt Learning Techniques: Evaluation of an Automated Analysis System Using Medical Text

Yan Zhuang iD, Junyan Zhang iD, Xiuxing Li iD, Chao Liu iD, Yue Yu iD, Wei Dong iD, Kunlun He iD

JMIR Medical Informatics · 2025

DOI: 10.2196/63020 ↗

At a glance

Journal article in JMIR Medical Informatics, 2025. Written by Yan Zhuang and 6 co-authors. Institution: Medical Big Data Research Center, Chinese PLA General Hospital, Beijing, China. Major MEDLINE topics: International Classification of Diseases, Machine Learning, Natural Language Processing. Cited 5 times so far (Crossref); 38 references. The full text is freely available in PubMed Central.

AbstractSubjectsDati bibliograficiCiteRelated studies

Abstract (original language)

Background

Machine learning models can reduce the burden on doctors by converting medical records into International Classification of Diseases (ICD) codes in real time, thereby enhancing the efficiency of diagnosis and treatment. However, it faces challenges such as small datasets, diverse writing styles, unstructured records, and the need for semimanual preprocessing. Existing approaches, such as naive Bayes, Word2Vec, and convolutional neural networks, have limitations in handling missing values and understanding the context of medical texts, leading to a high error rate. We developed a fully automated pipeline based on the Key–bidirectional encoder representations from transformers (BERT) approach and large-scale medical records for continued pretraining, which effectively converts long free text into standard ICD codes. By adjusting parameter settings, such as mixed templates and soft verbalizers, the model can adapt flexibly to different requirements, enabling task-specific prompt learning.

Objective

This study aims to propose a prompt learning real-time framework based on pretrained language models that can automatically label long free-text data with ICD-10 codes for cardiovascular diseases without the need for semiautomatic preprocessing.

Methods

We integrated 4 components into our framework: a medical pretrained BERT, a keyword filtration BERT in a functional order, a fine-tuning phase, and task-specific prompt learning utilizing mixed templates and soft verbalizers. This framework was validated on a multicenter medical dataset for the automated ICD coding of 13 common cardiovascular diseases (584,969 records). Its performance was compared against robustly optimized BERT pretraining approach, extreme language network, and various BERT-based fine-tuning pipelines. Additionally, we evaluated the framework’s performance under different prompt learning and fine-tuning settings. Furthermore, few-shot learning experiments were conducted to assess the feasibility and efficacy of our framework in scenarios involving small- to mid-sized datasets.

Results

Compared with traditional pretraining and fine-tuning pipelines, our approach achieved a higher micro–F1-score of 0.838 and a macro–area under the receiver operating characteristic curve (macro-AUC) of 0.958, which is 10% higher than other methods. Among different prompt learning setups, the combination of mixed templates and soft verbalizers yielded the best performance. Few-shot experiments showed that performance stabilized and the AUC peaked at 500 shots.

Conclusions

These findings underscore the effectiveness and superior performance of prompt learning and fine-tuning for subtasks within pretrained language models in medical practice. Our real-time ICD coding pipeline efficiently converts detailed medical free text into standardized labels, offering promising applications in clinical decision-making. It can assist doctors unfamiliar with the ICD coding system in organizing medical record information, thereby accelerating the medical process and enhancing the efficiency of diagnosis and treatment.

Subjects

MeSH terms (MEDLINE)

International Classification of DiseasesMachine LearningNatural Language ProcessingHumansElectronic Health RecordsCardiovascular Diseases

Author keywords

BERTICDInternational Classification of Diseasesbidirectional encoder representations from transformerscardiovascular diseasefew-shot learningmulticenter medical datapretrained language modelsprompt learning

Authors and institutions

  • Medical Big Data Research Center, Chinese PLA General Hospital, Beijing, China.
  • School of Computer Science & Technology, Beijing Institute of Technology, Beijing, China.
  • Digital Health China Technologies Co Ltd, Beijing, China.
  • Senior Department of Cardiology, The Sixth Medical Center of PLA General Hospital, Beijing, China.

ORCID profiles: Yan Zhuang, Junyan Zhang, Xiuxing Li, Chao Liu, Yue Yu, Wei Dong, Kunlun He

Cite

APA

Zhuang, Y., Zhang, J., Li, X., Liu, C., Yu, Y., Dong, W., & He, K. (2025). Autonomous International Classification of Diseases Coding Using Pretrained Language Models and Advanced Prompt Learning Techniques: Evaluation of an Automated Analysis System Using Medical Text. JMIR Medical Informatics. https://doi.org/10.2196/63020

Vancouver

Zhuang Y, Zhang J, Li X, Liu C, Yu Y, Dong W, et al. Autonomous International Classification of Diseases Coding Using Pretrained Language Models and Advanced Prompt Learning Techniques: Evaluation of an Automated Analysis System Using Medical Text. JMIR Medical Informatics. 2025. doi:10.2196/63020

BibTeX
@article{Zhuang2025,
  author = {Zhuang, Yan and Zhang, Junyan and Li, Xiuxing and Liu, Chao and Yu, Yue and Dong, Wei and He, Kunlun},
  title = {Autonomous International Classification of Diseases Coding Using Pretrained Language Models and Advanced Prompt Learning Techniques: Evaluation of an Automated Analysis System Using Medical Text},
  journal = {JMIR Medical Informatics},
  year = {2025},
  doi = {10.2196/63020}
}
Start for free →