EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling

Jinying Chen iD, Sarah L Cutrona iD, Ajay Dharod iD, Adam Moses iD, Aaron Bridges iD, Brian Ostasiewski iD, Kristie L Foley iD, Thomas K Houston iD, iDAPT (Implementation & Informatics Developing Adaptable Processes and Technologies for Cancer Control) Implementation Science Center for Cancer Control

JAMIA Open · 03.07.2025 · 8 (4)

DOI: 10.1093/jamiaopen/ooaf050 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 15.07.2025. Verfasst von Jinying Chen und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medicine/Section of Preventive Medicine and Epidemiology, Boston University Chobanian & Avedisian School of Medicine, Boston, MA 02118,…. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 84 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: NCATS NIH HHS, National Cancer Institute, National Institutes of Health.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

To develop and test a novel machine learning approach for monitoring impact of computerized clinical decision support (CDS) tools on clinicians’ electronic health record (EHR) activities. Materials and Methods Our CDS monitoring approach leverages topic modeling, a latent-variable statistical machine learning method, to infer health providers’ EHR activities from EHR audit logs. We applied this approach to monitor the impact of a tobacco cessation support CDS tool newly implemented in 5 cancer clinics (2018-2021). We trained the topic model on EHR audit log data from 3445 encounters (pre-CDS-implementation: 1734, post-CDS-implementation: 1711) for patients with active smoking status. The number of topics was automatically determined based on within-topic coherence and across-topic divergence, and the identified topics were assigned clinically relevant EHR activity labels by 4 domain experts.

Results

The topic model identified 2 distinct activities focusing on CDS (act on CDS, bypass/postpone CDS), 2 activities related to CDS (review patient records and address alerts, use note templates and acknowledge the completion of CDS), 6 related to accessing (access patient station) and reviewing patient data (external records, synopsis data, snapshot of patient data, problem list/diagnosis/notes, treatment plan), and 4 related to modifying EHR (modify diagnosis/problem lists, document visit with record review, perform administrative activities for visit and billing, and document follow-up care plan). Comparing matched 1-hour after-check-in windows post-implementation (n = 841) versus pre-implementation (n = 841) of CDS, the mean prevalence (expressed as proportions out of 1.0) of providers’ EHR-use activity increased on CDS-focused activities (0.073, 95% CI, 0.066-0.079) and CDS-related activities (0.098, 95% CI, 0.089-0.106) and decreased on modifying EHR (−0.113, 95% CI, −0.124 to −0.102) and reviewing patient data (−0.058, 95% CI, −0.072 to −0.044).

Discussion

Our topic model-based CDS monitoring approach can identify shifts in prevalence of EHR-use activities pre-implementation versus post-implementation. This approach can be applied to detect unintended changes in EHR activities on a large population scale following CDS implementation, providing valuable insights to guide focused qualitative investigations for CDS improvement or de-implementation.

Conclusion

Our approach offers a scalable, data-driven framework for evaluating the real-world impact of EHR-embedded CDS tools. Built on a generic machine learning framework, this approach could be adapted to explore impact of other healthcare quality improvement strategies using EHR-integrated CDS interventions.

Schlagwörter

Stichwörter der Autor:innen

EHR audit logsclinical decision supportelectronic alertsmonitoringtopic model

Autor:innen und Institutionen

  • Department of Medicine/Section of Preventive Medicine and Epidemiology, Boston University Chobanian & Avedisian School of Medicine, Boston, MA 02118, United States.
  • Data Science Core, Boston University Chobanian & Avedisian School of Medicine, Boston, MA 02118, United States.
  • Department of Internal Medicine/Section of General Internal Medicine, Wake Forest University School of Medicine, Winston Salem, NC 27101, United States.
  • Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA 01655, United States.
  • Center for Healthcare Organization and Implementation Research (CHOIR), VA Bedford Healthcare System, Bedford, MA 01730, United States.
  • Department of Implementation Science, Division of Public Health Sciences, Wake Forest University School of Medicine, Winston Salem, NC 27101, United States.

ORCID-Profile: Jinying Chen, Sarah L Cutrona, Ajay Dharod, Adam Moses, Aaron Bridges, Brian Ostasiewski, Kristie L Foley, Thomas K Houston

Förderung

  • NCATS NIH HHS
  • National Cancer Institute
  • National Institutes of Health

Zitieren

APA

Chen, J., Cutrona, S. L., Dharod, A., Moses, A., Bridges, A., Ostasiewski, B., Foley, K. L., Houston, T. K., & iDAPT (Implementation & Informatics Developing Adaptable Processes and Technologies for Cancer Control) Implementation Science Center for Cancer Control (2025). Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling. JAMIA Open, 8(4). https://doi.org/10.1093/jamiaopen/ooaf050

Vancouver

Chen J, Cutrona SL, Dharod A, Moses A, Bridges A, Ostasiewski B, et al. Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling. JAMIA Open. 2025;8(4). doi:10.1093/jamiaopen/ooaf050

BibTeX
@article{Chen2025,
  author = {Chen, Jinying and Cutrona, Sarah L and Dharod, Ajay and Moses, Adam and Bridges, Aaron and Ostasiewski, Brian and Foley, Kristie L and Houston, Thomas K and iDAPT (Implementation & Informatics Developing Adaptable Processes and Technologies for Cancer Control) Implementation Science Center for Cancer Control, iDAPT (Implementation & Informatics Developing Adaptable Processes and Technologies for Cancer Control) Implementation Science Center for Cancer Control},
  title = {Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling},
  journal = {JAMIA Open},
  year = {2025},
  volume = {8},
  number = {4},
  doi = {10.1093/jamiaopen/ooaf050}
}
Kostenlos starten →