Considerations

Won't students take each others' ideas? This is one reservation which is widely held by students. A widely-held concern - but Richard Milne (UCL Centre for Virology) comments on his own experience of setting up peer review activities, " 'I wasn't worried about students stealing each others' ideas ... you discuss a subject with somebody else and then formulate your own way of thinking about it based on the conversation you’ve had"'. Students can be encouraged to credit each others' ideas (and a convention can be agreed upon for circumstances of anonymity).

Can students at any level of knowledge carry out good peer reviews? In their meta-analysis comparing validity of tutor and student assessments, Potentially yes - a meta-analysis by Falchikov and Goldfinch (2009) could not find evidence found that peer assessment in higher level courses was any more reliable (between different assessors) or valid (according to a standard) than at introductory levels. They speculate that careful preparation by tutors and students can compensate for subject knowledge of students at earlier stages of their courseat introductory levels was as valid as at higher levels. They attribute this to good preparation. McConlogue (2014) suggests beginning early with low stakes assessment - perhaps the introduction of a draft essay - and building up student expertise over the duration of a programme.

Considerations corresponding to the settings above

Instructions. Students : At first students do tend to prefer tutor marking, and . They may assume that there is a correct mark for their work which is not open to interpretation (McConlogue, 2012). For this reason, avoid relying on textual instructions alone. Most researchers into peer assessment (including Bloxham and West, 2007; McConologue, 2014; Nicol, 2010; Orsmond, 2004; Topping, 2009) stress agree about the need to involve students in discussing - and ideally negotiating - the yardstick against which they will measure themselves and others, rather than relying on textual instructions alone. They recommend discussing others and themselves. It is important to discuss the rationale, criteria and expectations for peer and/or self review before, during and after the activity. Discussing or negotiating expectations could clarify clarifies how much time students were are expected to spend on each review and indicate how much feedback should be given, which will help . This helps to even out the quality and quantity of peer feedback and , which in turn helps to avoid perceptions of unfairness (Cartney, 2010) and increase acceptance of peer-assessment. Less time is needed for preparation once students are familiar with the process and ethos.

Dates.The fact that PeerMark is for formative feedback only : That PeerMark asks students to give feedback only (i.e. not a numeric mark) raises possibilities for students reviewing reviewing draft work. This can happen at an early, relatively unpolished stage which remains open to rewriting on the basis of feedback (Colvill, 2010). In which this case, set the Feedback Release Date to allow time for students to make changes in advance of their final credit-bearing submission. The time allowance for the PeerMark activity (i.e. between Start Date and Due Date) should reflect the time students are expected to spend , and allow for their other commitments.

Considerations

Award full points if the review is written. Where this . This means that students  get full marks if they fill in all parts of the peer review form, and no marks if they don't. As an all-or-nothing setting is deployed as it might be an incentive to participate, but do keep in mind the importance of dialogue at all stages.

Allow students to view author and reviewer names. Setting the peer review to be anonymous will prevent view author and reviewer names. Students are likely to experience some social discomfort about commenting on each others' work. Enabling anonymity helps to avoid the peer review being determined by friendship, enmity or power processes determining the review and forestall collusion, and disrupts any collusion among students. Clear criteria and an ethos which encourages mutual constructive criticism while discouraging platitudes are other measures to allay the social comfort students may feel about commenting on others' work. It may be necessary work out with students a convention for referencing each others' work in the absence of names, should they want to do so.

Allow submitters to read all papers after the Start Date. As well as allowing students to compare different work, this allows students to select work to review, if this has been enabled in the settingsThis allows students to view all the pieces of work as they are submitted.

Allow students to read ALL papers and ALL reviews after the Feedback Release Date. Again, this communicates to students that they are welcome and encouraged . This allows students to view all submissions and reviews after the peer review process has ended. Enabling this allows students to benchmark both their submissions and their reviews, and opens could open up the possibility of conversations which outlast the PeerMark activity.

Distribution of papers. Keep in mind boredom, tiredness and time pressures when deciding how many submissions each student should review. Falchikov and Goldfinch (2009) found didn't find that larger numbers of reviewers did not bring brought any validity gains and may . In fact they reported that large numbers might reduce reliability due to the 'diffusion of responsibility effect' whereby students are less likely to perceive their own review as mattering. (Falchikov and Goldfinch were comparing peer and tutor marks rather than feedback, though).

Require self-review. Since one of the aims of peer-assessment is to help students use the criteria in their own work, self-review is likely to would be a helpful exercise. However, because it may pose a distinct idiosyncratic or cultural set of complications related to self-esteem, self- confidence, modesty, and how students habitually estimate their own ability (Saito and Fujita, 2004) , it is a good idea to ask students to carry it out after they have completed their peer review(s). For this reason PeerMark requires at least one peer review, whether or not there is a self review.

least one peer review, even if there's a self review.

Considerations

Who decides

the questions?Peer assessment researchers strongly recommend involving students in developing and clarifying the assessment criteria

- even if they arrive at similar criteria to the tutors

(and of course the criteria are likely to be aligned to the module's intended learning outcomes) . Involving students in this way increases a sense of ownership,

reduces anxiety, and also

promotes a shared understanding about the meaning of the criteria

. As well as improving the reliability and validity

of the assessment, it also builds confidence in the process. Orsmond (2004, Section 2) discusses alternative techniques for introducing assessment criteria to students, including giving opportunities to practice

applying the criteria.

What kinds of questions? Falchikov and Goldfinch (2009) found that asking students to make a global judgement (i.e. of the submission as a whole) based on distinct criteria was more effective than either a judgement without criteria, or separate judgements of separate dimensions of the submission. McConlogue (2014) points out that as well as value judgements, reviewed students

also expect feedback that

signposts how they can improve; PeerMark

's open question type allows for these

.

Order of questions. Topping (2009) recommends asking students to give positive feedback first,  since this improves subsequent acceptance of negative feedback.

Opportunities for practice. Again,

practice is a clear recommendation from the

peer assessment literature and could be incorporated into the aforementioned recommended discussion of the criteria.

Considerations.

Does it matter which students review which other students' work? Tutors may want to connect students By default Turnitin makes random allocations, but if tutors want students to connect e.g. on the basis of interest. As well as matching students through PeerMark, another way to achieve this is to set up groups in your Moodle area and apply these to the Turnitin assignment. Alternatively, there are alternative ways to do this. Tutors can manually pair individual students through PeerMark. Or thirdly, PeerMark has the option of letting students choose the work they review (note though that this introduces the possibility that some students will receive feedback from more feedback sources than others - usually other students - which may be contentious).