Using PeerMark - guidance for staff
Contents
What is PeerMark?
PeerMark is a way for Moodle editors to set up a peer review for student submissions to Turnitin assignments which can exist before or alongside the standard tutor assessment. PeerMark is feedback only from the student reviewers i.e. no grade or numeric mark but reviewers receive a mark for their review.
The basic stages of a PeerMark activity are:
- Staff set up a Turnitin assignment to which students submit their work.
- Staff set up an associated PeerMark, including review questions, setting how many pieces of work each students review, by when, how allocated, whether anonymous &c.
- Students submit their work to the Turnitin assignment.
- Students review others' work.
- Students receive the feedback given by their peers, along with a mark for their own review(s).
- [[CHECK]] Depending on the timing, students can then incorporate this feedback into a final submission
What might PeerMark help with?
- Students' ability to understand and work with assessment criteria.
- Students' participation in the authentic academic practice of peer review.
- The possibility of feedback that is quicker, more individualised, and more plentiful than tutors are able to provide.
- Triangulation - peer review and tutor assessment (not to mention self assessment) triangulate each other, giving students perspective on their work and (if they are permitted to see how other students reviewed the same work they reviewed) on the criteria.
- Relatedly, insights into subjectivity and governance in the assessment process.
- Provision of feedback on students' draft work - given sufficient time for amendments before the deadline. (Worried students might take each others' ideas? See below.)
In the light of his research into peer assessment, Keith Topping (2009) suggests explaining to students,
"...that peer assessment involves students directly in learning, and should promote a sense of ownership, personal responsibility, and motivation. Teachers can also point out that peer assessment can increase variety and interest, activity and interactivity, identification and bonding, self-confidence, and empathy with others."
Considerations
Can students at any level of knowledge carry out good peer reviews? In their meta-analysis comparing validity of tutor and student assessments, Falchikov and Goldfinch (2009) could not find evidence that peer assessment in higher level courses was any more valid than at introductory levels. They speculate that careful preparation by tutors and students can compensate for subject knowledge of students at the early stages of their course.
Setting up a new PeerMark Assignment
First set up your Turnitin assignment as normal. |
|
In the Submission inbox for your assignment, click on the 'Launch Peermark Manager' icon |
|
If this is your first PeerMark assignment you can click on the 'create a new PeerMark assignment' link. |
|
PeerMark Assignment settings
In the 'Peermark Assignment' tab of the PeerMark Manager you enter basic information about the activity. Title This will appear for students and should be distinctive and descriptive. Point value (required) The marks available for the peer review itself - i.e. not for the reviewed work. Instructions to students Brief guidance about what students should do and why. Start date, Due date, Post date NB How do these relate to the Turnitin assignment's dates? Make sure you click the 'Save & Continue' button to proceed to the next tab.
Considerations Instructions. Students tend to prefer tutor marking, which may indicate positivist beliefs about objectivity in marking and the assumption that there is a correct mark for their work which is not open to interpretation (McConlogue, 2012). Most researchers into peer assessment including Topping (2009) stress the need to discuss with students the rationale for peer review in advance, rather than relying on instructions alone. |
|
|
|
Adding Questions
The 'PeerMark Questions' tab of the PeerMark Manager allows you create the questions you want the peer reviewers to answer. To add a question, click 'Add question' |
|
Enter your question text, the question type. There are two types of question you can use; |
|
For a 'Free response' question, enter the minimum answer length (this counts words). |
|
For a 'Scale' question, enter the scale size and the lowest and highest values |
|
You can also use libraries to manage your Peermark questions. Clicking on Library Settings allows you to create and delete libraries, and to save and retrieve questions from those libraries. There is also a 'Sample Library' which you can add pre-made questions from. |
|
Considerations
|
|
Distribution
Please note that after reviewing has started you won't be able to pair students - so do make any allocations in advance.
In the 'Distribution' tab of the PeerMark Manager you can see all the student accounts associated with this assignment and how they will be allocated reviews. If you want to, this is where you can get involved with who reviews whose work. If you can't see all the accounts you are expecting, click outside of the Peermark Manager to return to your Turnitin assignment page; then click its 'Turnitin Students' tab. From there you can click 'Enrol all students', which will bring in all students 'enrolled' in that Moodle course area. |
|
If you need to exempt student from the PeerMark activity, you can exclude them by clicking their adjacent red Minus icon; their name displays greyed-out and they gain a green Plus icon, which you can click if you need to reinstate them. If you want to pair students (so that a particular student is allocated the work of another particular student to review, overriding any other distribution settings) you can do so by clicking the blue Plus icon and then selecting a student to pair with from the dropdown list. Paired students are then required to review the work they are allocated. Considerations. Does it matter which students review which other students' work? You may want to connect students on the basis of interest. Another way to achieve this is to set up groups in your Moodle area and apply these to the Turnitin assignment. Topping (2009) suggests connecting students based on their ability. |
|
|
|
Additional Settings
On the 'Peermark Assignment' tab there is a link for additional settings. Here's some explanation of the less obvious ones. 'Award full points if review is written' If ticked this means tutors will not be able to mark the reviews and a student will need to meet set requirements for every part of the review in order to get the available marks, on an all-or-nothing basis. If unticked, tutors can assign and differentiate marks for each student's review. 'Allow students to view author and reviewer names' If left unticked, you probably need to remind students not to put any identifying information in the title, filename, or body of their work. 'Paper(s) automatically distributed by Peermark' This sets the number of randomly allocated papers each student has to review. 'Papers(s) selected by the student' This sets the number of papers a student can choose to review. Students can review a combination of allocated and selected papers. 'Require self-review' If checked, a student has to review their own paper. It isn't currently possible to select self review only - the number allocated by PeerMark has to be at least one. |
|
|
|
Accessing Peermark reviews
In the 'Submission Inbox' you can see details of all the PeerMark assignments set up for that Turnitin assignment. |
|
Click on the 'Launch Peermark Reviews' icon |
|
If there is more than one Peermark assignment set up for this Turnitin assignment you can select the one you want . |
|
The 'Reviews' tab shows you a list of the students. Students that have submitted a paper will have an icon next to them under the 'Review' column. The 'Received' column shows how many reviews a student's submission has received.The 'Submitted' column shows how many reviews a student has submitted. Clicking on the numbers with a grey background takes you to either the 'Received Reviews' or the 'Submitted Reviews' tab, and from there, clicking on the blue 'tick' icon launches the document viewer. Tutors can also write reviews. Clicking on the blue 'Write instructor review' icon in the 'Review' column allows you to write an additional review as the assignment tutor. If you have left a review for a submission it will display a green 'Edit instructor review' icon. |
|
|
|
The document viewer
At the top of the document you will see details of which paper you are viewing, and which student has reviewed it. |
|
On the right hand column you will see a column with two tabs. |
|
|
|
Marking reviews
Once the due date has passed, you can go into the document viewer for a particular review and enter a grade in the in the top right of the screen. |
|
HOW are grades released???? |
|
Tips
Saving – Apart from the 'Save & Continue' button on the Peermark Assessment tab, there are no other save buttons within the manager. To exit the manager you click out of the manager somewhere on the greyed out portion of the screen. |
|
|
|
|
|
Bugs
Sometimes, when switching between tabs, you will see an 'Error, you are not authorised to access this resource' message. Click away from the window and reopen it vis the Peermark manage icon. |
|
|
|
References
- Falchikov, N., & Goldfinch, J. (2000). Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks. Review of Educational Research, 70(3), 287–322. doi:10.3102/00346543070003287
- McConlogue, T. (2012). But is it fair? Developing students’ understanding of grading complex written work through peer assessment. Assessment & Evaluation in Higher Education, 37(1), 113–123. doi:10.1080/02602938.2010.515010
- Nicol, D., (2007). Peer Evaluation in Assessment Review project. Available from http://www.reap.ac.uk/PEER.aspx
- Sorensen, E., (2013). Experiences of using peer assessment in a 4th year design module. Available from: http://www.ucl.ac.uk/teaching-learning/case-studies-news/assessment-feedback/peer-assessment-chemical-engineering
- Topping, K. J. (2009). Peer Assessment. Theory Into Practice, 48(1), 20–27. doi:10.1080/00405840802577569
