Using PeerMark - guidance for staff
Contents
| Table of Contents | ||
|---|---|---|
|
...
"...that peer assessment involves students directly in learning, and should promote a sense of ownership, personal responsibility, and motivation. Teachers can also point out that peer assessment can increase variety and interest, activity and interactivity, identification and bonding, self-confidence, and empathy with others."
| Info |
|---|
Considerations Can students at any level of knowledge carry out good peer reviews? In their meta-analysis comparing validity of tutor and student assessments, Falchikov and Goldfinch (2009) could not find evidence that peer assessment in higher level courses was any more valid than at introductory levels. They speculate that careful preparation by tutors and students can compensate for subject knowledge of students at earlier stages of their course. Can peer assessment work in every subject area? Although they found some differences (arts, social sciences and medical sciences had lower peer-academic agreement in some cases) Falchikov and Goldfinch did not find that subject area had a significant effect on the quality of peer assessment. They also report that peer assessment of academic products (e.g. essays, posters) or processes (e.g. oral presentation skills, groupwork participation) have more validity than those in the context of professional practice. This may be related to students' greater experience with academic products and processes. Their research also suggests that while students are equal to peer-assessment in one new discipline, requiring multi-disciplinary assessments is likely to reduce validity. |
Setting up a new PeerMark Assignment
First set up your Turnitin assignment as normal. |
|
In the Submission inbox for your assignment, click on the 'Launch Peermark Manager' icon |
|
If this is your first PeerMark assignment you can click on the 'create a new PeerMark assignment' link. |
|
PeerMark Assignment settings
In the 'Peermark Assignment' tab of the PeerMark Manager you enter basic information about the activity.
Title
This will appear for students and should be distinctive and descriptive.
Point value (required)
The marks available for the peer review itself - i.e. not for the reviewed work. This reflects research findings that asking students to assign numeric marks to their peers exacerbates any sense of risk and brings undue complications and pressure to peer review without bringing any particular learning benefits.
Instructions to students
Brief guidance about what students should do and why.
Start date, Due date, Post date
NB How do these relate to the Turnitin assignment's dates?
Make sure you click the 'Save & Continue' button to proceed to the next tab.
| Info |
|---|
Considerations Instructions. Students tend to prefer tutor marking, which may indicate positivist beliefs about objectivity in marking and the assumption that there is a correct mark for their work which is not open to interpretation (McConlogue, 2012). Most researchers into peer assessment (including Bloxham and West, 2007; McConologue, 2014; Nicol, 2010; Topping, 2009) stress the need to discuss with students the rationale, criteria and expectations for peer review before, during and after the activity, rather than relying on textual instructions alone. Discussing or negotiating expectations could clarify how much time students were expected to spend on each review and an indication of how much feedback should be given. These particulars would help to even out the quality and quantity of peer feedback and avoid perceptions of unfairness (Cartney, 2010). Point value. This should be sufficient to indicate to the students that their participation in peer review matters. Dates. Since Turnitin is for formative feedback, PeerMark on draft work may be helpful, in which case set the Feedback Release Date to allow time for students to make changes in advance. The time allowance for the PeerMark activity (i.e. between Start Date and Due Date) should reflect the time students are expected to spend, and accommodate their other commitments. |
...
...
...
PeerMark Assigment - Additional Settings
...
On the 'Peermark Assignment' tab there is a link for additional settings. Here's some explanation of the less obvious ones.
'Award full points if review is written'
If ticked this means tutors will not be able to mark the reviews and a student will need to meet set requirements for every part of the review in order to get the available marks, on an all-or-nothing basis. If unticked, tutors can assign and differentiate marks for each student's review.
'Allow students to view author and reviewer names'
If left unticked, you probably need to remind students not to put any identifying information in the title, filename, or body of their work.
'Paper(s) automatically distributed by Peermark'
This sets the number of randomly allocated papers each student has to review.
'Papers(s) selected by the student'
This sets the number of papers a student can choose to review. Students can review a combination of allocated and selected papers.
'Require self-review'
If checked, a student has to review their own paper. It isn't currently possible to select self review only - the number allocated by PeerMark has to be at least one.
...
...
Won't students take each others' ideas? This is one reservation which is widely held by students. Richard Milne (UCL Centre for Virology) comments on his own experience of setting up peer review activities, "I wasn't worried about students stealing each others' ideas ... you discuss a subject with somebody else and then formulate your own way of thinking about it based on the conversation you’ve had". Students can be encouraged to credit each others' ideas (and a convention can be agreed for circumstances of anonymity). Can students at any level of knowledge carry out good peer reviews? In their meta-analysis comparing validity of tutor and student assessments, Falchikov and Goldfinch (2009) could not find evidence that peer assessment in higher level courses was any more reliable or valid than at introductory levels. They speculate that careful preparation by tutors and students can compensate for subject knowledge of students at earlier stages of their course. Can peer assessment work in every subject area? Although they found some differences (arts, social sciences and medical sciences had lower peer-academic agreement in some cases) Falchikov and Goldfinch did not find that subject area had a significant effect on the quality of peer assessment. They also report that peer assessment of academic products (e.g. essays, posters) or processes (e.g. oral presentation skills, groupwork participation) have more validity than those in the context of professional practice. This may be related to students' greater experience with academic products and processes. Their research also suggests that while students are equal to peer-assessment in one new discipline, requiring multi-disciplinary assessments is likely to reduce validity. |
Setting up a new PeerMark Assignment
First set up your Turnitin assignment as normal. |
|
In the Submission inbox for your assignment, click on the 'Launch Peermark Manager' icon |
|
If this is your first PeerMark assignment you can click on the 'create a new PeerMark assignment' link. |
|
PeerMark Assignment settings
In the 'Peermark Assignment' tab of the PeerMark Manager you enter basic information about the activity. Title This will appear for students and should be distinctive and descriptive. Point value (required) The marks available for the peer review itself - i.e. not for the reviewed work. This reflects research findings that asking students to assign numeric marks to their peers exacerbates any sense of risk and brings undue complications and pressure to peer review without bringing any particular learning benefits. Instructions to students Brief guidance about what students should do and why. Start date, Due date, Post date NB How do these relate to the Turnitin assignment's dates? Make sure you click the 'Save & Continue' button to proceed to the next tab.
|
| ||
|
|
PeerMark Assigment - Additional Settings
On the 'Peermark Assignment' tab there is a link for additional settings. Here's some explanation for the less obvious ones. 'Award full points if review is written' If ticked this means tutors will not be able to mark the reviews and a student will need to meet set requirements for every part of the review in order to get the available marks, on an all-or-nothing basis. If unticked, tutors can assign and differentiate marks for each student's review. 'Allow students to view author and reviewer names' If left unticked, you probably need to remind students not to put any identifying information in the title, filename, or body of their work. 'Paper(s) automatically distributed by Peermark' This sets the number of randomly allocated papers each student has to review. 'Papers(s) selected by the student' This sets the number of papers a student can choose to review. Students can review a combination of allocated and selected papers. 'Require self-review' If checked, a student has to review their own paper. It isn't currently possible to select self review only - the number allocated by PeerMark has to be at least one. |
| ||
|
|
Adding Questions
The 'PeerMark Questions' tab of the PeerMark Manager allows you create the questions you want the peer reviewers to answer. To add a question, click 'Add question' |
| ||
Enter your question text, the question type. There are two types of question you can use; |
| ||
For a 'Free response' question, enter the minimum answer length (this counts words). |
| ||
For a 'Scale' question, enter the scale size and the lowest and highest values |
| ||
You can also use libraries to manage your Peermark questions. Clicking on Library Settings allows you to create and delete libraries, and to save and retrieve questions from those libraries. There is also a 'Sample Library' which you can add pre-made questions from. |
| ||
|
|
...
References
- Bloxham, S., & West, A. (2007). Learning to write in higher education: students’ perceptions of an intervention in developing understanding of assessment criteria. Teaching in Higher Education, 12(1), 77–89.
- Cartney, P. (2010). Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used. Assessment & Evaluation in Higher Education, 35(5), 551–564.
- Falchikov, N., & Goldfinch, J. (2000). Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks. Review of Educational Research, 70(3), 287–322.
- McConlogue, T. (2012). But is it fair? Developing students’ understanding of grading complex written work through peer assessment. Assessment & Evaluation in Higher Education, 37(1), 113–123.
- McConlogue, T. (2014). Making judgements: investigating the process of composing and receiving peer feedback. Studies in Higher Education, 1–12.
- Milne, R. (2013). Peer review of virology essays. Available from: https://www.ucl.ac.uk/teaching-learning/case-studies-news/assessment-feedback/peer-review-of-virology-essays
- Nicol, D., (2007). Peer Evaluation in Assessment Review project. Available from http://www.reap.ac.uk/PEER.aspx
- Nicol, D. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35(5), 501–517.
- Schommer, M. (1990). Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology, 82(3), 498–504.
- Sorensen, E. (2013). Experiences of using peer assessment in a 4th year design module. Available from: http://www.ucl.ac.uk/teaching-learning/case-studies-news/assessment-feedback/peer-assessment-chemical-engineering
- Topping, K. J. (2009). Peer Assessment. Theory Into Practice, 48(1), 20–27.
- Yorke, M. (2003). Formative assessment in higher education: moves towards theory and the enhancement of pedagogic practice. Higher Education, 45, 477–501.
...