Crowdsourced vs. Traditional Usability Testing: A Comparative Usability Evaluation of Learning Management Systems
International Journal of Innovations in Science and Technology · 10.06.2026 · 1166
DOI: 10.33411/ijist/1866 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Innovations in Science and Technology, online erschienen am 10.06.2026. Verfasst von Sadiq Khan und 3 weiteren Autor:innen. Die Arbeit nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction/Importance of Study: Usability and User Experience (UX) evaluation is essential for improving Learning Management Systems (LMS) because these platforms directly influence students’ learning effectiveness and satisfaction.Novelty statement: This study uniquely compares traditional user testing with a crowdsourced usability testing method on prominent global LMS platforms (Google Classroom, Canvas, and Edredo), addressing a critical research gap in large-scale benchmarking and resource-efficiency.Material and Method: An experimental research design involved 15 traditional and 15 crowdsourced testers who performedidentical tasks. Sessions were recorded via OBS software and analyzed using the Molich and Dumas coding scheme. To determine statistical significance between the two methods, the Mann-Whitney U test was employed to evaluate effectiveness (issue counts), efficiency (time), and cost-effectiveness.Result and Discussion: The evaluation identified a total of 310 usability issues. The traditional method discovered 170 (55%), while the crowdsourced method identified 140 (45%). In terms of effectiveness parity, thetraditional method detected 48 key issues compared to 45 inthe crowdsourced approach(p=0.23014), proving no significant difference in identifying critical failures. However, crowdsourced testing demonstrated extreme economic efficiency, reducing costs by 6.4x—expending $150 compared to $960 for traditional testing (p=0.00001). Regarding time efficiency, crowdsourced sessions were completed in 4.88 hours compared to 6.07 hours for traditional methods (p=0.11123), indicating comparable operational speeds.Concluding Remarks: Crowdsourced usability testing provides a statistically validated alternative to traditional methods, achieving comparable issue detection effectiveness while significantly reducing resource expenditure.
Zitieren
APA
Khan, S., Nasir, M., Jalil, Z., & Afzal, Q. (2026). Crowdsourced vs. Traditional Usability Testing: A Comparative Usability Evaluation of Learning Management Systems. International Journal of Innovations in Science and Technology, 1166. https://doi.org/10.33411/ijist/1866
Vancouver
Khan S, Nasir M, Jalil Z, Afzal Q. Crowdsourced vs. Traditional Usability Testing: A Comparative Usability Evaluation of Learning Management Systems. International Journal of Innovations in Science and Technology. 2026:1166. doi:10.33411/ijist/1866
BibTeX
@article{Khan2026,
author = {Khan, Sadiq and Nasir, Muhammad and Jalil, Zakia and Afzal, Qaisar},
title = {Crowdsourced vs. Traditional Usability Testing: A Comparative Usability Evaluation of Learning Management Systems},
journal = {International Journal of Innovations in Science and Technology},
year = {2026},
pages = {1166},
doi = {10.33411/ijist/1866}
}
EUVIMED