EUVIMEDObjevte EUVIMED
Systematische ÜbersichtOpen access

Developing a Comprehensive List of Criteria to Evaluate the Characteristics and Quality of eHealth Smartphone Apps: Systematic Review

Entwicklung einer umfassenden Liste von Kriterien zur Bewertung der Eigenschaften und Qualität von eHealth-Smartphone-Apps: systematische Übersichtsarbeit

Janette Ribaut iD, Annette DeVito Dabbs iD, Fabienne Dobbels iD, Alexandra Teynor iD, Elisabeth Veronica Mess iD, Theresa Hoffmann iD, Sabina De Geest iD

JMIR mHealth and uHealth · 2024-01-15 · 12 · e48625

DOI: 10.2196/48625 ↗

Auf einen Blick

Die Übersichtsarbeit identifizierte 216 Kriterien zur Bewertung von eHealth-Smartphone-Apps. Die Autor:innen leiteten daraus ein Rahmenkonzept mit 6 Bewertungsdimensionen ab.

Fragestellung

Welche veröffentlichten Kriterien zur Beschreibung und Bewertung von eHealth-Smartphone-Apps sind verfügbar?

Methode

Systematische Übersichtsarbeit; Datenbanken und Studienregister wurden durchsucht; eingeschlossen waren Volltexte in 6 Sprachen. Aus den Artikeln wurde ein Konzeptrahmen abgeleitet und iterativ diskutiert.

Ergebnisse

1258 Artikel gescreent, 128 eingeschlossen. Daraus wurden 205 Kriterien identifiziert; nach Konsens 12 umbenannt und 11 ergänzt, insgesamt 216 Kriterien. 43 Instrumente wurden nur einmal genutzt.

Einschränkungen

83/128 Artikel berichteten keine theoretischen Leitlinien; viele Kriteriensets hatten keine konsistenten Definitionen und Terminologie.

Schlussfolgerung der Autor:innen

Die Studie liefert einen umfassenden Überblick über aktuell genutzte Kriterien zur Beschreibung und Bewertung von Apps. Das Ergebnis ist für die klinische Praxis in der vorliegenden Form noch unpraktisch.

Kernaussagen

  • 216 Bewertungskriterien für eHealth-Smartphone-Apps
  • 6 Dimensionen: Kontext, Stakeholder, Features, Entwicklung, Umsetzung, Evaluation
  • Bedarf an einem theoriegeleiteten, praxistauglichen Bewertungstool

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Systematische Übersicht in JMIR mHealth and uHealth, published online on 2024-01-15. Written by Janette Ribaut and 6 co-authors. Institution: Institute of Nursing Science, Department Public Health, University of Basel, Basel, Switzerland. Major MEDLINE topics: Mobile Applications, Telemedicine. Cited 24 times so far (Crossref); 101 references. The full text is freely available in PubMed Central.

Auf einen BlickAbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

The field of eHealth is growing rapidly and chaotically. Health care professionals need guidance on reviewing and assessing health-related smartphone apps to propose appropriate ones to their patients. However, to date, no framework or evaluation tool fulfills this purpose.

Objective

Before developing a tool to help health care professionals assess and recommend apps to their patients, we aimed to create an overview of published criteria to describe and evaluate health apps.

Methods

We conducted a systematic review to identify existing criteria for eHealth smartphone app evaluation. Relevant databases and trial registers were queried for articles. Articles were included that (1) described tools, guidelines, dimensions, or criteria to evaluate apps, (2) were available in full text, and (3) were written in English, French, German, Italian, Portuguese, or Spanish. We proposed a conceptual framework for app evaluation based on the dimensions reported in the selected articles. This was revised iteratively in discussion rounds with international stakeholders. The conceptual framework was used to synthesize the reported evaluation criteria. The list of criteria was discussed and refined by the research team.

Results

Screening of 1258 articles yielded 128 (10.17%) that met the inclusion criteria. Of these 128 articles, 30 (23.4%) reported the use of self-developed criteria and described their development processes incompletely. Although 43 evaluation instruments were used only once, 6 were used in multiple studies. Most articles (83/128, 64.8%) did not report following theoretical guidelines; those that did noted 37 theoretical frameworks. On the basis of the selected articles, we proposed a conceptual framework to explore 6 app evaluation dimensions: context, stakeholder involvement, features and requirements, development processes, implementation, and evaluation. After standardizing the definitions, we identified 205 distinct criteria. Through consensus, the research team relabeled 12 of these and added 11 more—mainly related to ethical, legal, and social aspects—resulting in 216 evaluation criteria. No criteria had to be moved between dimensions.

Conclusions

This study provides a comprehensive overview of criteria currently used in clinical practice to describe and evaluate apps. This is necessary as no reviewed criteria sets were inclusive, and none included consistent definitions and terminology. Although the resulting overview is impractical for use in clinical practice in its current form, it confirms the need to craft it into a purpose-built, theory-driven tool. Therefore, in a subsequent step, based on our current criteria set, we plan to construct an app evaluation tool with 2 parts: a short section (including 1-3 questions/dimension) to quickly disqualify clearly unsuitable apps and a longer one to investigate more likely candidates in closer detail. We will use a Delphi consensus-building process and develop a user manual to prepare for this undertaking. Trial Registration PROSPERO International Prospective Register of Systematic Reviews CRD42021227064; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021227064

Subjects

MeSH terms (MEDLINE)

Mobile ApplicationsTelemedicineHumansConsensusDatabases, Factual

Author keywords

decision-makingmobile appsmobile phoneprogram evaluationsmartphonesystematic reviewtelemedicine

Authors and institutions

  • Institute of Nursing Science, Department Public Health, University of Basel, Basel, Switzerland.
  • Department of Hematology, University Hospital Basel, Basel, Switzerland.
  • School of Nursing, Department of Acute & Tertiary Care, University of Pittsburgh, Pittsburgh, PA, United States.
  • Clinical Translational Science Institute, University of Pittsburgh, Pittsburgh, PA, United States.
  • Academic Center for Nursing and Midwifery, Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium.
  • Department of Computer Science, University of Applied Sciences, Augsburg, Germany.

ORCID profiles: Janette Ribaut, Annette DeVito Dabbs, Fabienne Dobbels, Alexandra Teynor, Elisabeth Veronica Mess, Theresa Hoffmann, Sabina De Geest

Cite

APA

Ribaut, J., DeVito Dabbs, A., Dobbels, F., Teynor, A., Mess, E. V., Hoffmann, T., & De Geest, S. (2024). Developing a Comprehensive List of Criteria to Evaluate the Characteristics and Quality of eHealth Smartphone Apps: Systematic Review. JMIR mHealth and uHealth, 12, e48625. https://doi.org/10.2196/48625

Vancouver

Ribaut J, DeVito Dabbs A, Dobbels F, Teynor A, Mess EV, Hoffmann T, et al. Developing a Comprehensive List of Criteria to Evaluate the Characteristics and Quality of eHealth Smartphone Apps: Systematic Review. JMIR mHealth and uHealth. 2024;12:e48625. doi:10.2196/48625

BibTeX
@article{Ribaut2024,
  author = {Ribaut, Janette and DeVito Dabbs, Annette and Dobbels, Fabienne and Teynor, Alexandra and Mess, Elisabeth Veronica and Hoffmann, Theresa and De Geest, Sabina},
  title = {Developing a Comprehensive List of Criteria to Evaluate the Characteristics and Quality of eHealth Smartphone Apps: Systematic Review},
  journal = {JMIR mHealth and uHealth},
  year = {2024},
  volume = {12},
  pages = {e48625},
  doi = {10.2196/48625}
}
Start for free →