EUVIMEDEUVIMED entdecken
Mixed-Methods-StudieOpen Access

Analyzing Patient Complaints in Web-Based Reviews of Private Hospitals in Selangor, Malaysia, Using Large Language Model–Assisted Content Analysis: Mixed Methods Study

Muhammad Hafiz Sulaiman iD, Nora Muda iD, Fatimah Abdul Razak iD

JMIR Formative Research · 2025

DOI: 10.2196/69075 ↗

Kurz gefasst

Mixed-Methods-Studie in der Zeitschrift JMIR Formative Research, 2025. Verfasst von Muhammad Hafiz Sulaiman und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Mathematical Sciences, Faculty of Science and Technology, National University of Malaysia, Bangi, Malaysia. Hauptthemen laut MEDLINE: Internet, Hospitals, Private, Patient Satisfaction. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language model (LLM)–assisted content analysis (LACA) is a modification of traditional content analysis, leveraging the LLM to codevelop codebooks and automatically assign thematic codes to a web-based reviews dataset.

Objective

This study aims to develop and validate the use of LACA for analyzing hospital web-based reviews and to identify themes of issues from web-based reviews using this method.

Methods

Web-based reviews for 53 private hospitals in Selangor, Malaysia, were acquired. Fake reviews were filtered out using natural language processing and machine learning algorithms trained on yelp.com validated datasets. GPT-4o mini model application programming interface (API) was then applied to filter out reviews without any quality issues. In total, 200 of the remaining reviews were randomly extracted and fed into the GPT-4o mini model API to produce a codebook validated through parallel human-LLM coding to establish interrater reliability. The codebook was then used to code (label) all reviews in the dataset. The thematic codes were then summarized into themes using factor analysis to increase interpretability.

Results

A total of 14,938 web-based reviews were acquired, of which 1121 (9.3%) were fake, 1279 (12%) contained negative sentiments, and 9635 (88%) did not contain any negative sentiment. GPT-4o mini model subsequently inducted 41 thematic codes together with their definitions. Average human-GPT interrater reliability is perfect (κ=0.81). Factor analysis identified 6 interpretable latent factors: “Service and Communication Effectiveness,” “Clinical Care and Patient Experience,” “Facilities and Amenities Quality,” “Appointment and Patient Flow,” “Financial and Insurance Management,” and “Patient Rights and Accessibility.” The cumulative explained variance for the six factors is 0.74, and Cronbach α is between 0.88 and 0.97 (good and excellent) for all factors except factor 6 (0.61: questionable). The factors identified follow a global pattern of issues identified from the literature.

Conclusions

A data collection and processing pipeline consisting of Python Selenium, the GPT-4o mini model API, and a factor analysis module can support valid and reliable thematic analysis. Despite the potential for collection and information bias in web-based reviews, LACA of web-based reviews is cost-effective, time-efficient, and can be performed in real time, helping hospital managers develop hypotheses for further investigations promptly.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

InternetHospitals, PrivatePatient SatisfactionMalaysiaHumansReproducibility of ResultsNatural Language ProcessingLarge Language Models

Stichwörter der Autor:innen

big datahospital qualitylarge language modelpatient satisfactionweb-based review

Autor:innen und Institutionen

  • Department of Mathematical Sciences, Faculty of Science and Technology, National University of Malaysia, Bangi, Malaysia.
  • Quality Unit, Clinical Management Section, Hospital Sultan Idris Shah, Serdang, Malaysia.

ORCID-Profile: Muhammad Hafiz Sulaiman, Nora Muda, Fatimah Abdul Razak

Zitieren

APA

Sulaiman, M. H., Muda, N., & Abdul Razak, F. (2025). Analyzing Patient Complaints in Web-Based Reviews of Private Hospitals in Selangor, Malaysia, Using Large Language Model–Assisted Content Analysis: Mixed Methods Study. JMIR Formative Research. https://doi.org/10.2196/69075

Vancouver

Sulaiman MH, Muda N, Abdul Razak F. Analyzing Patient Complaints in Web-Based Reviews of Private Hospitals in Selangor, Malaysia, Using Large Language Model–Assisted Content Analysis: Mixed Methods Study. JMIR Formative Research. 2025. doi:10.2196/69075

BibTeX
@article{Sulaiman2025,
  author = {Sulaiman, Muhammad Hafiz and Muda, Nora and Abdul Razak, Fatimah},
  title = {Analyzing Patient Complaints in Web-Based Reviews of Private Hospitals in Selangor, Malaysia, Using Large Language Model–Assisted Content Analysis: Mixed Methods Study},
  journal = {JMIR Formative Research},
  year = {2025},
  doi = {10.2196/69075}
}
Kostenlos starten →