EUVIMEDOpdag EUVIMED
Randomisierte kontrollierte StudieVergleichsstudieOpen access

Comparison of Two Symptom Checkers (Ada and Symptoma) in the Emergency Department: Randomized, Crossover, Head-to-Head, Double-Blinded Study

Johannes Knitza iD, Ragip Hasanaj iD, Jonathan Beyer iD, Franziska Ganzer iD, Anna Slagman iD, Myrto Bolanaki iD, Hendrik Napierala iD, Malte L Schmieding iD, Nizam Al-Zaher iD, Till Orlemann iD, Felix Muehlensiepen iD, Julia Greenfield iD and 5 more

Journal of Medical Internet Research · 2024

DOI: 10.2196/56514 ↗

At a glance

Randomisierte kontrollierte Studie in Journal of Medical Internet Research, 2024. Written by Johannes Knitza and 16 co-authors. Institution: Institute for Digital Medicine, University Hospital Giessen, Philipps University, Marburg, Germany. Major MEDLINE topics: Emergency Service, Hospital, Triage. Cited 14 times so far (Crossref); 41 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Emergency departments (EDs) are frequently overcrowded and increasingly used by nonurgent patients. Symptom checkers (SCs) offer on-demand access to disease suggestions and recommended actions, potentially improving overall patient flow. Contrary to the increasing use of SCs, there is a lack of supporting evidence based on direct patient use.

Objective

This study aimed to compare the diagnostic accuracy, safety, usability, and acceptance of 2 SCs, Ada and Symptoma.

Methods

A randomized, crossover, head-to-head, double-blinded study including consecutive adult patients presenting to the ED at University Hospital Erlangen. Patients completed both SCs, Ada and Symptoma. The primary outcome was the diagnostic accuracy of SCs. In total, 6 blinded independent expert raters classified diagnostic concordance of SC suggestions with the final discharge diagnosis as (1) identical, (2) plausible, or (3) diagnostically different. SC suggestions per patient were additionally classified as safe or potentially life-threatening, and the concordance of Ada’s and physician-based triage category was assessed. Secondary outcomes were SC usability (5-point Likert-scale: 1=very easy to use to 5=very difficult to use) and SC acceptance net promoter score (NPS).

Results

A total of 450 patients completed the study between April and November 2021. The most common chief complaint was chest pain (160/437, 37%). The identical diagnosis was ranked first (or within the top 5 diagnoses) by Ada and Symptoma in 14% (59/437; 27%, 117/437) and 4% (16/437; 13%, 55/437) of patients, respectively. An identical or plausible diagnosis was ranked first (or within the top 5 diagnoses) by Ada and Symptoma in 58% (253/437; 75%, 329/437) and 38% (164/437; 64%, 281/437) of patients, respectively. Ada and Symptoma did not suggest potentially life-threatening diagnoses in 13% (56/437) and 14% (61/437) of patients, respectively. Ada correctly triaged, undertriaged, and overtriaged 34% (149/437), 13% (58/437), and 53% (230/437) of patients, respectively. A total of 88% (385/437) and 78% (342/437) of participants rated Ada and Symptoma as very easy or easy to use, respectively. Ada’s NPS was –34 (55% [239/437] detractors; 21% [93/437] promoters) and Symptoma’s NPS was –47 (63% [275/437] detractors and 16% [70/437]) promoters.

Conclusions

Ada demonstrated a higher diagnostic accuracy than Symptoma, and substantially more patients would recommend Ada and assessed Ada as easy to use. The high number of unrecognized potentially life-threatening diagnoses by both SCs and inappropriate triage advice by Ada was alarming. Overall, the trustworthiness of SC recommendations appears questionable. SC authorization should necessitate rigorous clinical evaluation studies to prevent misdiagnoses, fatal triage advice, and misuse of scarce medical resources. Trial Registration German Register of Clinical Trials DRKS00024830; https://drks.de/search/en/trial/DRKS00024830

Subjects

MeSH terms (MEDLINE)

Emergency Service, HospitalTriageAdultAgedFemaleHumansMaleMiddle AgedCross-Over StudiesDouble-Blind Method

Author keywords

apps, health service researchdecision support systemdiagnostic accuracyeHealthemergencysymptom checkertriage

Authors and institutions

  • Institute for Digital Medicine, University Hospital Giessen, Philipps University, Marburg, Germany.
  • Department of Internal Medicine 3, Friedrich-Alexander University Erlangen-Nürnberg, Universitätsklinikum Erlangen, Erlangen, Germany.
  • Deutsches Zentrum für Immuntherapie, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany.
  • Université Grenoble Alpes, Grenoble, France.
  • Department of Cardiology, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany.
  • Emergency and Acute Medicine and Health Services Research in Emergency Medicine, Charité - Universitätsmedizin Berlin, Berlin, Germany.

ORCID profiles: Johannes Knitza, Ragip Hasanaj, Jonathan Beyer, Franziska Ganzer, Anna Slagman, Myrto Bolanaki, Hendrik Napierala, Malte L Schmieding, Nizam Al-Zaher, Till Orlemann, Felix Muehlensiepen, Julia Greenfield, Nicolas Vuillerme, Sebastian Kuhn, Georg Schett, Stephan Achenbach, Katharina Dechant

Cite

APA

Knitza, J., Hasanaj, R., Beyer, J., Ganzer, F., Slagman, A., Bolanaki, M., Napierala, H., Schmieding, M. L., Al-Zaher, N., Orlemann, T., Muehlensiepen, F., Greenfield, J., Vuillerme, N., Kuhn, S., Schett, G., Achenbach, S., & Dechant, K. (2024). Comparison of Two Symptom Checkers (Ada and Symptoma) in the Emergency Department: Randomized, Crossover, Head-to-Head, Double-Blinded Study. Journal of Medical Internet Research. https://doi.org/10.2196/56514

Vancouver

Knitza J, Hasanaj R, Beyer J, Ganzer F, Slagman A, Bolanaki M, et al. Comparison of Two Symptom Checkers (Ada and Symptoma) in the Emergency Department: Randomized, Crossover, Head-to-Head, Double-Blinded Study. Journal of Medical Internet Research. 2024. doi:10.2196/56514

BibTeX
@article{Knitza2024,
  author = {Knitza, Johannes and Hasanaj, Ragip and Beyer, Jonathan and Ganzer, Franziska and Slagman, Anna and Bolanaki, Myrto and Napierala, Hendrik and Schmieding, Malte L and Al-Zaher, Nizam and Orlemann, Till and Muehlensiepen, Felix and Greenfield, Julia and Vuillerme, Nicolas and Kuhn, Sebastian and Schett, Georg and Achenbach, Stephan and Dechant, Katharina},
  title = {Comparison of Two Symptom Checkers (Ada and Symptoma) in the Emergency Department: Randomized, Crossover, Head-to-Head, Double-Blinded Study},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  doi = {10.2196/56514}
}
Start for free →