How we are making peer assessment more useful and fair: Five tips for MOOC organisers
In our last blog post, we explained how we studied and compared the scores and feedback given through expert and peer assessments in our TeachUP courses. Now that we established their differences, in this blog we turn to the question of fairness and usefulness, as perceived by course participants. We will also cover what we are doing in the European Schoolnet Academy with these results to improve peer feedback.
Largely, both forms of assessment were perceived as useful and fair. 84% of participants said both peer and expert assessments were useful.
Looking at fairness, however, both types of feedback were perceived quite differently: 96% of participants said they felt their peer assessment was fair, while only 79% said they felt expert assessment was fair. As established, qualitative and quantitative feedback from peers was generally higher and more positive than expert feedback. It is plausible that participants perceived the feedback as fairer because it was more positive. But we don’t know if this is indeed the case. Another explanation could be that the expert assessments were anonymous, while the peer assessments were not.
Finally, 77% of the course participants said they didn’t prefer one assessment method over the other. While participants received more actionable suggestions from the experts, assessing your peers’ work is also a great way to solidify your knowledge on the course content. So, peer assessments are there to stay.
How to make (peer) assessments more useful?
Based on the detailed analysis of the data, the researchers formulated 6 generic recommendations that any MOOC organiser can use to improve their course. Let’s have a look at five concrete points to make peer assessment (more) useful for you:
- Provide a clear ‘assessment tool’, such as an evaluation rubric with clear descriptions for each grade. Rubrics should include clear descriptors, or even examples, of each scoring category. This will allow for more consistent feedback overall.
- Give participants the possibility to respond to the feedback they received. This will help emphasize and enable peer dialogue (for example about the teaching beliefs apparent in a coursework).
- Train course participants on how to give feedback, for example with optional practice peer assessment. In the end participants will receive more detailed and constructive suggestions that help improve one’s lesson plan.
- During the MOOC, some time should be spent learning how to give quality peer feedback. For example, providing examples of good qualitative feedback can inspire participants to give more actionable items in their feedback.
- In addition to peer assessment, provide expert assessment for a small random selection of the course assignments. This way course participants get to see, in a hands-on manner, how reviews could be made more detailed and concrete.
Which of the key suggestions from the report might be implemented differs from provider to provider. Random expert assessments, for example, would require a certain investment in time. At the European Schoolnet Academy we are currently exploring the feasibility of implementing some of these points.
What is European Schoolnet Academy doing now?
But we are already acting on some of the report’s recommendations! We have been providing detailed rubrics for quite some time already, and a few years ago we created a leaflet with tips for giving constructive feedback (see link below).
Following the report, we also started testing an optional practice peer assessments. In last year’s Integrated STEM Teaching MOOCs we offered participants the practice as an optional step. The goal was to assess whether participants who participated in the practice felt better equipped to assess their peers’ work.
For these courses 313 participants indicated in the evaluation survey that they had used the practice peer assessment. Overall, teachers that joined the practice found it useful: 81% of the respondents indicated that it helped them assess their peers’ learning scenario in the final module. Following this positive feedback, we decided to incorporate a practice peer assessment in our recent course on active learning. And we’ll continue to offer practice peer assessments in future courses.
How to give constructive feedback
If you would like more immediate resources on how to give proper feedback, check out our leaflet on how to provide constructive feedback.
Are you interested in reading more about the research on peer and expert assessment? Be sure to have a look at the TeachUP report Peer versus expert assessment – How to make assessment in online teacher training work.
Now that you’ve heard about both peer and expert assessment, which one would you prefer to receive? Share your thoughts in the comments!