EUVIMEDEUVIMED entdecken
Crossref

Analysis of the reliability of the evaluation and scoringprocess for abstract submissions for the “JournéesFrancophones de la Kinésithérapie 2023”

Jean-Philippe Deneuville, Matthieu Gallou-Guyot, Matthieu Guémann

European Rehabilitation Journal · 2024

DOI: 10.52057/erj.v4i1.46 ↗

Kurz gefasst

Fachartikel in der Zeitschrift European Rehabilitation Journal, 2024. Verfasst von Jean-Philippe Deneuville und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The French Society of Physiotherapy (SFP) organises biennial conferences known as the "JournéesFrancophones de Kinésithérapie" (JFK) since 2007. Abstracts are submitted and are evaluated for acceptance by twoindependent reviewers from the SFP using a predefined rating checklist. However, the reliability of this process has neverbeen evaluated.

Objective

This study aims to assess the inter-rater reliability and internal consistency of the JFK submissionrating process conducted by the scientific committee.

Methods

Blind reviewers evaluated each submission in pairs using astandardized 47-item rating checklist, categorised into five domains including background, method, results and relevance tophysiotherapy. Reliability was assessed using an Intraclass Correlation Coefficient (ICC) and Cohen’s Kappa. Agreement wasassessed using the Standard Error of Measurement (SEM), Coefficient of Variation, Bland-Altman analysis, and percentageof agreement. Internal consistency was assessed using Cronbach’s Alpha.

Results

36 reviewers assessed a total of 217abstracts. The reliability, measured by ICC, was poor (0.39 [CI95% = 0.30; 0.49]) as was the agreement; SEM = 3.08 andCoefficient of Variation = 23.1%. All individual checklist items had a Cohen’s kappa coeff below 0.6. All but one domainhad a Cronbach Alpha above 0.7, indicating good consistency. However, five domains had a Cronbach Alpha above 0.9,suggesting redundancy.

Conclusion

The JFK 2023 submission rating process displayed poor reliability. These findings canguide improvements in creating the JFK 2025 checklist. This study may help future scientific committees to enhance theirevaluation process.

Zitieren

APA

Deneuville, J., Gallou-Guyot, M., & Guémann, M. (2024). Analysis of the reliability of the evaluation and scoringprocess for abstract submissions for the “JournéesFrancophones de la Kinésithérapie 2023”. European Rehabilitation Journal. https://doi.org/10.52057/erj.v4i1.46

Vancouver

Deneuville J, Gallou-Guyot M, Guémann M. Analysis of the reliability of the evaluation and scoringprocess for abstract submissions for the “JournéesFrancophones de la Kinésithérapie 2023”. European Rehabilitation Journal. 2024. doi:10.52057/erj.v4i1.46

BibTeX
@article{Deneuville2024,
  author = {Deneuville, Jean-Philippe and Gallou-Guyot, Matthieu and Guémann, Matthieu},
  title = {Analysis of the reliability of the evaluation and scoringprocess for abstract submissions for the “JournéesFrancophones de la Kinésithérapie 2023”},
  journal = {European Rehabilitation Journal},
  year = {2024},
  doi = {10.52057/erj.v4i1.46}
}
Kostenlos starten →