EUVIMEDDescoperă EUVIMED
Open access

Synthetic Data-Driven Approaches for Chinese Medical Abstract Sentence Classification: Computational Study

Jiajia Li iD, Zikai Wang iD, Longxuan Yu iD, Hui Liu iD, Haitao Song iD

JMIR Formative Research · 2025

DOI: 10.2196/54803 ↗

At a glance

Journal article in JMIR Formative Research, 2025. Written by Jiajia Li and 4 co-authors. Institution: Shanghai Artificial Intelligence Research Institute Co., Ltd, Shanghai, China. Major MEDLINE topics: Abstracting and Indexing, Medicine, Chinese Traditional, Natural Language Processing. Cited 1 times so far (Crossref); 40 references. The full text is freely available in PubMed Central.

RezumatSubjectsDate bibliograficeCiteRelated studies

Abstract (original language)

Background

Medical abstract sentence classification is crucial for enhancing medical database searches, literature reviews, and generating new abstracts. However, Chinese medical abstract classification research is hindered by a lack of suitable datasets. Given the vastness of Chinese medical literature and the unique value of traditional Chinese medicine, precise classification of these abstracts is vital for advancing global medical research.

Objective

This study aims to address the data scarcity issue by generating a large volume of labeled Chinese abstract sentences without manual annotation, thereby creating new training datasets. Additionally, we seek to develop more accurate text classification algorithms to improve the precision of Chinese medical abstract classification.

Methods

We developed 3 training datasets (dataset #1, dataset #2, and dataset #3) and a test dataset to evaluate our model. Dataset #1 contains 15,000 abstract sentences translated from the PubMed dataset into Chinese. Datasets #2 and #3, each with 15,000 sentences, were generated using GPT-3.5 from 40,000 Chinese medical abstracts in the CSL database. Dataset #2 used titles and keywords for pseudolabeling, while dataset #3 aligned abstracts with category labels. The test dataset includes 87,000 sentences from 20,000 abstracts. We used SBERT embeddings for deeper semantic analysis and evaluated our model using clustering (SBERT-DocSCAN) and supervised methods (SBERT-MEC). Extensive ablation studies and feature analyses were conducted to validate the model’s effectiveness and robustness.

Results

Our experiments involved training both clustering and supervised models on the 3 datasets, followed by comprehensive evaluation using the test dataset. The outcomes demonstrated that our models outperformed the baseline metrics. Specifically, when trained on dataset #1, the SBERT-DocSCAN model registered an impressive accuracy and F 1 -score of 89.85% on the test dataset. Concurrently, the SBERT-MEC algorithm exhibited comparable performance with an accuracy of 89.38% and an identical F 1 -score. Training on dataset #2 yielded similarly positive results for the SBERT-DocSCAN model, achieving an accuracy and F 1 -score of 89.83%, while the SBERT-MEC algorithm recorded an accuracy of 86.73% and an F 1 -score of 86.51%. Notably, training with dataset #3 allowed the SBERT-DocSCAN model to attain the best with an accuracy and F 1 -score of 91.30%, whereas the SBERT-MEC algorithm also showed robust performance, obtaining an accuracy of 90.39% and an F 1 -score of 90.35%. Ablation analysis highlighted the critical role of integrated features and methodologies in improving classification efficiency.

Conclusions

Our approach addresses the challenge of limited datasets for Chinese medical abstract classification by generating novel datasets. The deployment of SBERT-DocSCAN and SBERT-MEC models significantly enhances the precision of classifying Chinese medical abstracts, even when using synthetic datasets with pseudolabels.

Subjects

MeSH terms (MEDLINE)

Abstracting and IndexingMedicine, Chinese TraditionalNatural Language ProcessingHumansAlgorithmsChinaDatabases, Factual

Author keywords

Chinese medicalaccuracyalgorithmdatasetdeep learningefficiencyglobal medical researchlarge language modelsmedical abstract sentence classificationrobustnesssynthetic datasetstraditional Chinese medicine

Authors and institutions

  • Shanghai Artificial Intelligence Research Institute Co., Ltd, Shanghai, China.
  • Xiangfu Laboratory, Jiaxing, China.
  • School of Chemistry and Chemical Engineering, Shanghai Jiao Tong University, Shanghai, China.
  • Inner Mongolia Academy of Science and Technology, Hohhot, China.
  • University of California San Diego, San Diego, CA, United States.
  • Shanghai Civil Aviation College, Shanghai, China.

ORCID profiles: Jiajia Li, Zikai Wang, Longxuan Yu, Hui Liu, Haitao Song

Cite

APA

Li, J., Wang, Z., Yu, L., Liu, H., & Song, H. (2025). Synthetic Data-Driven Approaches for Chinese Medical Abstract Sentence Classification: Computational Study. JMIR Formative Research. https://doi.org/10.2196/54803

Vancouver

Li J, Wang Z, Yu L, Liu H, Song H. Synthetic Data-Driven Approaches for Chinese Medical Abstract Sentence Classification: Computational Study. JMIR Formative Research. 2025. doi:10.2196/54803

BibTeX
@article{Li2025,
  author = {Li, Jiajia and Wang, Zikai and Yu, Longxuan and Liu, Hui and Song, Haitao},
  title = {Synthetic Data-Driven Approaches for Chinese Medical Abstract Sentence Classification: Computational Study},
  journal = {JMIR Formative Research},
  year = {2025},
  doi = {10.2196/54803}
}
Start for free →