EUVIMEDObjevte EUVIMED
Mixed-Methods-StudieOpen access

Usability and Acceptance Testing of an Electronic Patient-Reported Outcome Symptom Monitoring System for People Receiving Immune Checkpoint Inhibitors: Mixed Methods Study

Julia Lai-Kwon iD, Claudia Rutherford iD, Stephanie Best iD, Hasan Shahid Ferdous iD, Iris Zhang iD, Thai-Khoa Ly iD, Dishan Herath iD, Kate Burbury iD, Michael Jefford iD

JMIR Formative Research · 2026-03-18 · 10 · e79694-e79694

DOI: 10.2196/79694 ↗

At a glance

Mixed-Methods-Studie in JMIR Formative Research, published online on 2026-03-18. Written by Julia Lai-Kwon and 8 co-authors. Institution: Department of Medical Oncology, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia. Major MEDLINE topics: Patient Reported Outcome Measures, Immune Checkpoint Inhibitors, Patient Acceptance of Health Care. Cited 2 times so far (Crossref); 47 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Immune checkpoint inhibitors are widely used in oncology but can cause immune-related adverse events (irAEs), which may be severe or life-threatening if not detected early. Electronic patient-reported outcome (ePRO) symptom monitoring systems may facilitate timely recognition and management of irAEs. Usability testing is a critical stage in ePRO system development, yet no published examples of formal usability and acceptance testing exist.

Objective

This study aims to assess the usability and acceptance of a co-designed ePRO symptom monitoring prototype for irAEs embedded within the Epic electronic medical record.

Methods

Testing was conducted at an Australian quaternary cancer center. Eligible participants were patients who had received or were receiving immune checkpoint inhibitors, their caregivers, or clinicians (oncologists and nurse specialists). Participants completed baseline digital literacy assessments (16-item Mobile Device Proficiency Questionnaire [MDPQ-16] and 12-item Computer Proficiency Questionnaire [CPQ-12]) before a structured testing session. Each session involved role-specific tasks using the patient-facing Health Hub or the clinician-facing Epic electronic medical record. Usability was assessed using the System Usability Scale (SUS). Acceptance was assessed using a customized Unified Theory of Acceptance and Use of Technology (UTAUT) questionnaire. Semistructured interviews were used to capture qualitative feedback.

Results

A total of 30 participants (7 patients, 3 caregivers, 10 oncologists, and 10 nurse specialists) completed 10 testing sessions. Median MDPQ-16 and CPQ-12 scores were higher for clinicians compared to patients and caregivers. Median SUS scores indicated high usability—patients and caregivers: 77.5% (IQR 70.0%‐86.3%), oncologists: 82.5% (IQR 80.0%‐90.0%), and nurse specialists: 80.0% (IQR 75.6%‐94.4%). Median UTAUT scores demonstrated strong user acceptance—patients or caregivers: 4.27 (IQR 4.09‐4.58), oncologists: 4.33 (IQR 4‐4.63), and nurse specialists: 4.23 (IQR 3.87‐4.57). Health Hub usability themes highlighted overall ease of navigation and efficiency of reporting, but a need for clearer survey navigation, simplification of the actions page, and improved organization of trend graphs. For clinicians, themes included efficient side effect capture and intuitive system design, but a need to improve navigation to results, optimize data display, and facilitate team-based alert management. Health Hub acceptance themes highlighted patient empowerment to self-manage, enhanced patient-clinician communication, and reinforcement of existing care. However, concerns were raised about digital equity for vulnerable groups. Clinicians reported that the system streamlined side effect management between visits, aligned with existing Epic workflows, and could be tailored to personal preferences. Concerns remained regarding additional workload and medico-legal responsibilities associated with real-time alerts.

Conclusions

The ePRO prototype demonstrated high levels of usability and acceptance across patients, caregivers, and clinicians. Limitations around navigation and data visualization, alongside equity and workload concerns, will guide refinements prior to implementation. These findings emphasize the value of rigorous formative usability and acceptance testing to optimize ePRO systems prior to deployment in routine cancer care.

Subjects

MeSH terms (MEDLINE)

Patient Reported Outcome MeasuresImmune Checkpoint InhibitorsPatient Acceptance of Health CareHumansFemaleMaleMiddle AgedAgedAustraliaAdultSurveys and QuestionnairesElectronic Health RecordsNeoplasmsDigital Health

Author keywords

acceptancedigital healthelectronic patient-reported outcomesimmune checkpoint inhibitorsimmunotherapypatient-reported outcomessymptom monitoringusability

Substances

Immune Checkpoint Inhibitors

Authors and institutions

  • Department of Medical Oncology, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia.
  • Department of Health Services Research, Peter MacCallum Cancer Centre, 305 Grattan St, Melbourne, Victoria, 3000, Australia, 61 385595000.
  • Sydney Quality of Life Office, Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia.
  • Melbourne School of Health Sciences, University of Melbourne, Melbourne, Victoria, Australia.
  • Australian Genomics, Murdoch Children's Research Institute, Melbourne, Victoria, Australia.
  • Centre for Digital Transformation of Health, University of Melbourne, Melbourne, Victoria, Australia.

ORCID profiles: Julia Lai-Kwon, Claudia Rutherford, Stephanie Best, Hasan Shahid Ferdous, Iris Zhang, Thai-Khoa Ly, Dishan Herath, Kate Burbury, Michael Jefford

Cite

APA

Lai-Kwon, J., Rutherford, C., Best, S., Ferdous, H. S., Zhang, I., Ly, T. K., Herath, D., Burbury, K., & Jefford, M. (2026). Usability and Acceptance Testing of an Electronic Patient-Reported Outcome Symptom Monitoring System for People Receiving Immune Checkpoint Inhibitors: Mixed Methods Study. JMIR Formative Research, 10, e79694-e79694. https://doi.org/10.2196/79694

Vancouver

Lai-Kwon J, Rutherford C, Best S, Ferdous HS, Zhang I, Ly TK, et al. Usability and Acceptance Testing of an Electronic Patient-Reported Outcome Symptom Monitoring System for People Receiving Immune Checkpoint Inhibitors: Mixed Methods Study. JMIR Formative Research. 2026;10:e79694-e79694. doi:10.2196/79694

BibTeX
@article{LaiKwon2026,
  author = {Lai-Kwon, Julia and Rutherford, Claudia and Best, Stephanie and Ferdous, Hasan Shahid and Zhang, Iris and Ly, Thai-Khoa and Herath, Dishan and Burbury, Kate and Jefford, Michael},
  title = {Usability and Acceptance Testing of an Electronic Patient-Reported Outcome Symptom Monitoring System for People Receiving Immune Checkpoint Inhibitors: Mixed Methods Study},
  journal = {JMIR Formative Research},
  year = {2026},
  volume = {10},
  pages = {e79694--e79694},
  doi = {10.2196/79694}
}
Start for free →