Analysis of the reliability of the evaluation and scoringprocess for abstract submissions for the “JournéesFrancophones de la Kinésithérapie 2023”
European Rehabilitation Journal · 2024
DOI: 10.52057/erj.v4i1.46 ↗Kurz gefasst
Fachartikel in der Zeitschrift European Rehabilitation Journal, 2024. Verfasst von Jean-Philippe Deneuville und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
Background
The French Society of Physiotherapy (SFP) organises biennial conferences known as the "JournéesFrancophones de Kinésithérapie" (JFK) since 2007. Abstracts are submitted and are evaluated for acceptance by twoindependent reviewers from the SFP using a predefined rating checklist. However, the reliability of this process has neverbeen evaluated.
Objective
This study aims to assess the inter-rater reliability and internal consistency of the JFK submissionrating process conducted by the scientific committee.
Methods
Blind reviewers evaluated each submission in pairs using astandardized 47-item rating checklist, categorised into five domains including background, method, results and relevance tophysiotherapy. Reliability was assessed using an Intraclass Correlation Coefficient (ICC) and Cohen’s Kappa. Agreement wasassessed using the Standard Error of Measurement (SEM), Coefficient of Variation, Bland-Altman analysis, and percentageof agreement. Internal consistency was assessed using Cronbach’s Alpha.
Results
36 reviewers assessed a total of 217abstracts. The reliability, measured by ICC, was poor (0.39 [CI95% = 0.30; 0.49]) as was the agreement; SEM = 3.08 andCoefficient of Variation = 23.1%. All individual checklist items had a Cohen’s kappa coeff below 0.6. All but one domainhad a Cronbach Alpha above 0.7, indicating good consistency. However, five domains had a Cronbach Alpha above 0.9,suggesting redundancy.
Conclusion
The JFK 2023 submission rating process displayed poor reliability. These findings canguide improvements in creating the JFK 2025 checklist. This study may help future scientific committees to enhance theirevaluation process.
Zitieren
APA
Deneuville, J., Gallou-Guyot, M., & Guémann, M. (2024). Analysis of the reliability of the evaluation and scoringprocess for abstract submissions for the “JournéesFrancophones de la Kinésithérapie 2023”. European Rehabilitation Journal. https://doi.org/10.52057/erj.v4i1.46
Vancouver
Deneuville J, Gallou-Guyot M, Guémann M. Analysis of the reliability of the evaluation and scoringprocess for abstract submissions for the “JournéesFrancophones de la Kinésithérapie 2023”. European Rehabilitation Journal. 2024. doi:10.52057/erj.v4i1.46
BibTeX
@article{Deneuville2024,
author = {Deneuville, Jean-Philippe and Gallou-Guyot, Matthieu and Guémann, Matthieu},
title = {Analysis of the reliability of the evaluation and scoringprocess for abstract submissions for the “JournéesFrancophones de la Kinésithérapie 2023”},
journal = {European Rehabilitation Journal},
year = {2024},
doi = {10.52057/erj.v4i1.46}
}
EUVIMED