The best-achieving Online Students are overrepresented in Course Ratings

Tejeiro, Ricardo; Whitelock-Wainwright, Arthur; Perez, Alina; Urbina-Garcia, Angel

Abstract

Student ratings are the most used and influential measure of performance in Higher Education, and an integral component of formative and summative decision making. This may be particularly relevant in the relatively new online courses, where the pedagogical model is still developing. However, student ratings face strong controversy, and some remarkable challenges –one of which stems from the fact that not all students provide ratings. Nonresponse bias, or the lack of representativeness of the providers of ratings, has been measured and discussed in traditional courses, but to date no study has analysed nonresponse bias in the online evaluation of a fully online higher education course. Our study aims to close this gap. We analysed archival data for the students completing the intake module of four psychology online postgraduate programmes in a 2-year period (June 2014 to May 2016; n = 457). Statistical analyses included correlation, chi-square test, Mantel-Haenszel test of trend, Mann-Whitney’s U and regression analysis; effect size was measured with odds radios, Cramer´s V, and r. We found that the likelihood of providing ratings was not associated with sex, age, educational background, or familiarity with the British higher education system; however, respondents presented significantly higher values than nonrespondents in the key variable used to measure their learning experience –final mark. The implications of this finding are discussed in relation to Groves’ (2006) causal models for nonresponse bias, as well as the validity and leniency hypotheses.

Citation

Tejeiro, R., Whitelock-Wainwright, A., Perez, A., & Urbina-Garcia, A. (2018). The best-achieving Online Students are overrepresented in Course Ratings. European Journal of Open Education and e-Learning Studies, 3(2), 43-58. https://doi.org/10.5281/zenodo.1456188

• Journal Article Type: Article
• Acceptance Date: Sep 1, 2018
• Online Publication Date: Oct 1, 2018
• Publication Date: Oct 1, 2018
• Deposit Date: Sep 6, 2019
• Publicly Available Date: Sep 6, 2019
• Journal: European Journal of Open Education and e-Learning Studies
• Print ISSN: 2501-9120
• Publisher: Open Access Publishing Group
• Peer Reviewed: Peer Reviewed
• Volume: 3
• Issue: 2
• Pages: 43-58
• DOI: https://doi.org/10.5281/zenodo.1456188
• Keywords: Student ratings; Learning analytics; Teaching quality; Nonresponse; Online education
• Public URL: https://hull-repository.worktribe.com/output/1972082
• Publisher URL: https://oapub.org/edu/index.php/ejoe/article/view/1993
• Contract Date: Sep 6, 2019