EUVIMEDOdkrijte EUVIMED
ValidierungsstudieOpen access

An Advanced Machine Learning Model for a Web-Based Artificial Intelligence–Based Clinical Decision Support System Application: Model Development and Validation Study

Tai-Han Lin iD, Hsing-Yi Chung iD, Ming-Jr Jian iD, Chih-Kai Chang iD, Cherng-Lih Perng iD, Guo-Shiou Liao iD, Jyh-Cherng Yu iD, Ming-Shen Dai iD, Cheng-Ping Yu iD, Hung-Sheng Shang iD

Journal of Medical Internet Research · 2024

DOI: 10.2196/56022 ↗

At a glance

Validierungsstudie in Journal of Medical Internet Research, 2024. Written by Tai-Han Lin and 9 co-authors. Institution: Division of Clinical Pathology, Department of Pathology, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan. Major MEDLINE topics: Decision Support Systems, Clinical, Machine Learning, Breast Neoplasms, Artificial Intelligence. Cited 9 times so far (Crossref); 18 references. The full text is freely available in PubMed Central.

PovzetekSubjectsBibliografski podatkiCiteRelated studies

Abstract (original language)

Background

Breast cancer is a leading global health concern, necessitating advancements in recurrence prediction and management. The development of an artificial intelligence (AI)–based clinical decision support system (AI-CDSS) using ChatGPT addresses this need with the aim of enhancing both prediction accuracy and user accessibility.

Objective

This study aims to develop and validate an advanced machine learning model for a web-based AI-CDSS application, leveraging the question-and-answer guidance capabilities of ChatGPT to enhance data preprocessing and model development, thereby improving the prediction of breast cancer recurrence.

Methods

This study focused on developing an advanced machine learning model by leveraging data from the Tri-Service General Hospital breast cancer registry of 3577 patients (2004-2016). As a tertiary medical center, it accepts referrals from four branches—3 branches in the northern region and 1 branch on an offshore island in our country—that manage chronic diseases but refer complex surgical cases, including breast cancer, to the main center, enriching our study population’s diversity. Model training used patient data from 2004 to 2012, with subsequent validation using data from 2013 to 2016, ensuring comprehensive assessment and robustness of our predictive models. ChatGPT is integral to preprocessing and model development, aiding in hormone receptor categorization, age binning, and one-hot encoding. Techniques such as the synthetic minority oversampling technique address the imbalance of data sets. Various algorithms, including light gradient-boosting machine, gradient boosting, and extreme gradient boosting, were used, and their performance was evaluated using metrics such as the area under the curve, accuracy, sensitivity, and F1-score.

Results

The light gradient-boosting machine model demonstrated superior performance, with an area under the curve of 0.80, followed closely by the gradient boosting and extreme gradient boosting models. The web interface of the AI-CDSS tool was effectively tested in clinical decision-making scenarios, proving its use in personalized treatment planning and patient involvement.

Conclusions

The AI-CDSS tool, enhanced by ChatGPT, marks a significant advancement in breast cancer recurrence prediction, offering a more individualized and accessible approach for clinicians and patients. Although promising, further validation in diverse clinical settings is recommended to confirm its efficacy and expand its use.

Subjects

MeSH terms (MEDLINE)

Decision Support Systems, ClinicalMachine LearningBreast NeoplasmsArtificial IntelligenceInternetHumansFemaleMiddle AgedAdultAged

Author keywords

ChatGPTartificial intelligence–based clinical decision support systembreast cancer recurrencemachine learningpersonalized treatment planningpredictive model accuracy

Authors and institutions

  • Division of Clinical Pathology, Department of Pathology, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan.
  • Division of General Surgery, Department of Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan.
  • Division of Oncology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei City, Taiwan.
  • Department of Pathology, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan.

ORCID profiles: Tai-Han Lin, Hsing-Yi Chung, Ming-Jr Jian, Chih-Kai Chang, Cherng-Lih Perng, Guo-Shiou Liao, Jyh-Cherng Yu, Ming-Shen Dai, Cheng-Ping Yu, Hung-Sheng Shang

Cite

APA

Lin, T. H., Chung, H. Y., Jian, M. J., Chang, C. K., Perng, C. L., Liao, G. S., Yu, J. C., Dai, M. S., Yu, C. P., & Shang, H. S. (2024). An Advanced Machine Learning Model for a Web-Based Artificial Intelligence–Based Clinical Decision Support System Application: Model Development and Validation Study. Journal of Medical Internet Research. https://doi.org/10.2196/56022

Vancouver

Lin TH, Chung HY, Jian MJ, Chang CK, Perng CL, Liao GS, et al. An Advanced Machine Learning Model for a Web-Based Artificial Intelligence–Based Clinical Decision Support System Application: Model Development and Validation Study. Journal of Medical Internet Research. 2024. doi:10.2196/56022

BibTeX
@article{Lin2024,
  author = {Lin, Tai-Han and Chung, Hsing-Yi and Jian, Ming-Jr and Chang, Chih-Kai and Perng, Cherng-Lih and Liao, Guo-Shiou and Yu, Jyh-Cherng and Dai, Ming-Shen and Yu, Cheng-Ping and Shang, Hung-Sheng},
  title = {An Advanced Machine Learning Model for a Web-Based Artificial Intelligence–Based Clinical Decision Support System Application: Model Development and Validation Study},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  doi = {10.2196/56022}
}
Start for free →