EUVIMEDObjevte EUVIMED
Open access

Evaluating the Usability of an Emergency Department After Visit Summary: Staged Heuristic Evaluation

Hanna J Barton iD, Megan E Salwei iD, Rachel A Rutkowski iD, Kathryn Wust iD, Sheryl Krause iD, Peter Lt Hoonakker iD, Paula vW Dail iD, Denise M Buckley iD, Alexis Eastman iD, Brad Ehlenfeldt iD, Brian W Patterson iD, Manish N Shah iD and 3 more

JMIR Human Factors · 2023-03-09 · 10 · e43729

DOI: 10.2196/43729 ↗

At a glance

Journal article in JMIR Human Factors, published online on 2023-03-09. Written by Hanna J Barton and 14 co-authors. Institution: Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, Madison, WI, United States. Cited 12 times so far (Crossref); 25 references. The full text is freely available in PubMed Central. Funded by: AHRQ HHS, NCATS NIH HHS.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Heuristic evaluations, while commonly used, may inadequately capture the severity of identified usability issues. In the domain of health care, usability issues can pose different levels of risk to patients. Incorporating diverse expertise (eg, clinical and patient) in the heuristic evaluation process can help assess and address potential negative impacts on patient safety that may otherwise go unnoticed. One document that should be highly usable for patients—with the potential to prevent adverse outcomes—is the after visit summary (AVS). The AVS is the document given to a patient upon discharge from the emergency department (ED), which contains instructions on how to manage symptoms, medications, and follow-up care.

Objective

This study aims to assess a multistage method for integrating diverse expertise (ie, clinical, an older adult care partner, and health IT) with human factors engineering (HFE) expertise in the usability evaluation of the patient-facing ED AVS.

Methods

We conducted a three-staged heuristic evaluation of an ED AVS using heuristics developed for use in evaluating patient-facing documentation. In stage 1, HFE experts reviewed the AVS to identify usability issues. In stage 2, 6 experts of varying expertise (ie, emergency medicine physicians, ED nurses, geriatricians, transitional care nurses, and an older adult care partner) rated each previously identified usability issue on its potential impact on patient comprehension and patient safety. Finally, in stage 3, an IT expert reviewed each usability issue to identify the likelihood of successfully addressing the issue.

Results

In stage 1, we identified 60 usability issues that violated a total of 108 heuristics. In stage 2, 18 additional usability issues that violated 27 heuristics were identified by the study experts. Impact ratings ranged from all experts rating the issue as “no impact” to 5 out of 6 experts rating the issue as having a “large negative impact.” On average, the older adult care partner representative rated usability issues as being more significant more of the time. In stage 3, 31 usability issues were rated by an IT professional as “impossible to address,” 21 as “maybe,” and 24 as “can be addressed.” Conclusions Integrating diverse expertise when evaluating usability is important when patient safety is at stake. The non-HFE experts, included in stage 2 of our evaluation, identified 23% (18/78) of all the usability issues and, depending on their expertise, rated those issues as having differing impacts on patient comprehension and safety. Our findings suggest that, to conduct a comprehensive heuristic evaluation, expertise from all the contexts in which the AVS is used must be considered. Combining those findings with ratings from an IT expert, usability issues can be strategically addressed through redesign. Thus, a 3-staged heuristic evaluation method offers a framework for integrating context-specific expertise efficiently, while providing practical insights to guide human-centered design.

Subjects

Author keywords

discharge summarydocumentationemergencyemergency medicineheuristicheuristic evaluationhuman factors engineeringpatient safetysafetyusability

Authors and institutions

  • Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, Madison, WI, United States.
  • Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI, United States.
  • Center for Research and Innovation in Systems Safety, Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, TN, United States.
  • Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States.
  • School of Nursing, University of Wisconsin-Madison, Madison, WI, United States.
  • University of Wisconsin-Madison Health Sciences Patient and Family Advisory Council Member, Madison, WI, United States.

ORCID profiles: Hanna J Barton, Megan E Salwei, Rachel A Rutkowski, Kathryn Wust, Sheryl Krause, Peter Lt Hoonakker, Paula vW Dail, Denise M Buckley, Alexis Eastman, Brad Ehlenfeldt, Brian W Patterson, Manish N Shah, Barbara J King, Nicole E Werner, Pascale Carayon

Funding

  • AHRQ HHS
  • NCATS NIH HHS

Cite

APA

Barton, H. J., Salwei, M. E., Rutkowski, R. A., Wust, K., Krause, S., Hoonakker, P. L., Dail, P. V., Buckley, D. M., Eastman, A., Ehlenfeldt, B., Patterson, B. W., Shah, M. N., King, B. J., Werner, N. E., & Carayon, P. (2023). Evaluating the Usability of an Emergency Department After Visit Summary: Staged Heuristic Evaluation. JMIR Human Factors, 10, e43729. https://doi.org/10.2196/43729

Vancouver

Barton HJ, Salwei ME, Rutkowski RA, Wust K, Krause S, Hoonakker PL, et al. Evaluating the Usability of an Emergency Department After Visit Summary: Staged Heuristic Evaluation. JMIR Human Factors. 2023;10:e43729. doi:10.2196/43729

BibTeX
@article{Barton2023,
  author = {Barton, Hanna J and Salwei, Megan E and Rutkowski, Rachel A and Wust, Kathryn and Krause, Sheryl and Hoonakker, Peter Lt and Dail, Paula vW and Buckley, Denise M and Eastman, Alexis and Ehlenfeldt, Brad and Patterson, Brian W and Shah, Manish N and King, Barbara J and Werner, Nicole E and Carayon, Pascale},
  title = {Evaluating the Usability of an Emergency Department After Visit Summary: Staged Heuristic Evaluation},
  journal = {JMIR Human Factors},
  year = {2023},
  volume = {10},
  pages = {e43729},
  doi = {10.2196/43729}
}
Start for free →