Identifying healthcare needs with patient experience reviews using ChatGPT
PLOS ONE · 18.03.2025 · 20 (3) · e0313442
DOI: 10.1371/journal.pone.0313442 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 18.03.2025. Verfasst von Jiaxuan Li und 7 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Applied Sciences, Macao Polytechnic University, Macao, China. Hauptthemen laut MEDLINE: Data Mining, Health Services Needs and Demand, Patient Satisfaction. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Macao Science and Technology Development Fund, Macao Polytechnic University.
Abstract (Originalsprache)
Background
Valuable findings can be obtained through data mining in patients’ online reviews. Also identifying healthcare needs from the patient’s perspective can more accurately improve the quality of care and the experience of the visit. Thereby avoiding unnecessary waste of health care resources. The large language model (LLM) can be a promising tool due to research that demonstrates its outstanding performance and potential in directions such as data mining, healthcare management, and more.
Objective
We aim to propose a methodology to address this problem, specifically, the recent breakthrough of LLM can be leveraged for effectively understanding healthcare needs from patient experience reviews.
Methods
We used 504,198 reviews collected from a large online medical platform, haodf.com. We used the reviews to create Aspect Based Sentiment Analysis (ABSA) templates, which categorized patient reviews into three categories, reflecting the areas of concern of patients. With the introduction of thought chains, we embedded ABSA templates into the prompts for ChatGPT, which was then used to identify patient needs.
Results
Our method has a weighted total precision of 0.944, which was outstanding compared to the direct narrative tasks in ChatGPT-4o, which have a weighted total precision of 0.890. Weighted total recall and F1 scores also reached 0.884 and 0.912 respectively, surpassing the 0.802 and 0.843 scores for “direct narratives in ChatGPT.” Finally, the accuracy of the three sampling methods was 91.8%, 91.7%, and 91.2%, with an average accuracy of over 91.5%.
Conclusions
Combining ChatGPT with ABSA templates can achieve satisfactory results in analyzing patient reviews. As our work applies to other LLMs, we shed light on understanding the demands of patients and health consumers with novel models, which can contribute to the agenda of enhancing patient experience and better healthcare resource allocations effectively.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Faculty of Applied Sciences, Macao Polytechnic University, Macao, China.
- Department of Rehabilitation Medicine, The First Affiliated Hospital, Sun Yat-Sen University, Guangzhou, China.
- State University of New York, Songdo, Korea.
ORCID-Profile: Patrick Cheong-Iao Pang, Yapeng Wang
Förderung
- Macao Science and Technology Development Fund
- Macao Polytechnic University
Zitieren
APA
Li, J., Yang, Y., Chen, R., Zheng, D., Pang, P. C., Lam, C. K., Wong, D., & Wang, Y. (2025). Identifying healthcare needs with patient experience reviews using ChatGPT. PLOS ONE, 20(3), e0313442. https://doi.org/10.1371/journal.pone.0313442
Vancouver
Li J, Yang Y, Chen R, Zheng D, Pang PC, Lam CK, et al. Identifying healthcare needs with patient experience reviews using ChatGPT. PLOS ONE. 2025;20(3):e0313442. doi:10.1371/journal.pone.0313442
BibTeX
@article{Li2025,
author = {Li, Jiaxuan and Yang, Yunchu and Chen, Rong and Zheng, Dashun and Pang, Patrick Cheong-Iao and Lam, Chi Kin and Wong, Dennis and Wang, Yapeng},
title = {Identifying healthcare needs with patient experience reviews using ChatGPT},
journal = {PLOS ONE},
year = {2025},
volume = {20},
number = {3},
pages = {e0313442},
doi = {10.1371/journal.pone.0313442}
}
EUVIMED