Guidance on anonymous marking in Turnitin - including second marking, moderating and external examining
| Note |
|---|
| This guide is for Turnitin Assignment. If you are using Moodle Assignment, please see our separate guidance. |
| Anchor | ||||
|---|---|---|---|---|
|
On this page:
| Table of Contents |
|---|
Introduction
The UCL Board of Examiners of 26th February 2014 resolved '[t]hat a flexible and simple framework should be drafted which sought to prevent allegations of bias and outlined clear definitions of acceptable standards and good practice in this area...'
...
| Info |
|---|
This guidance is bound to evolve in line with policy, technologies and ideas from departments. You are invited to contact Digital Education with any questions or suggestions - Digital Education will amend this guidance accordingly and keep the E-Learning Champions informed of important improvements as they arise. |
| Top of page |
Terms
- Anonymous marking: see below for a definition and explanation.
- Bulk-download: downloading the contents of a submission inbox for marking outside Turnitin.
- Candidate number: a unique identifier (four letters and one digit) which students receive annually.
- Combinations of blind/open, full/sampled second marking: refer to the UCL Marking Policy*.
- External examining: refer to the UCL Marking Policy.
- Grademark: Turnitin's own marking environment.
- Intermediary: somebody without marking responsibilities, perhaps in an administrative or technical role.
- Markings: all notes, specific and general comments, and numeric marks made by markers.
- Moderating: refer to the UCL Marking Policy.
- Moodle Assignment: Moodle offers a separate online assessment environment. Its marking and feedback possibilities currently differ from Turnitin's. Only Turnitin offers an Originality Report, and Turnitin's suite of online annotation tools is slightly more developed.
- Paper ID: a numeric identifier unique to each paper, generated by Turnitin, but currently irrelevant outside Turnitin.
- Post Date: point at which marks and feedback are released to students (at this point student anonymity is lifted).
- Proxy identifier: see below for a definition and explanation.
- Student Record Number (SRN): this appears on the front of every student's ID card.
- Submission filename: the name of the file(s) submitted; this becomes important in cases where students' work is downloaded for marking outside Turnitin.
- Submission title: the title students type into Turnitin when submitting their work. This displays in the Turnitin inbox and can be used to allocate submissions to subject specialist markers.
- turnitinuk.com: allows access (where there are existing permissions) to a Turnitin submission inbox independently of Moodle; affords downloading of Grademarked work and - though only after the Post Date - original unmarked work. Please note Digital Education is unable to offer technical support for your activities on turnitinuk.com - however, Turnitin does offer support.
- Window of opportunity: a phrase used in this guidance to refer to a period just after Grademarking (if used) is complete and just before the Post Date de-anonymises students. During this window the still-anonymous, Grademarked submissions, complete with markings, can be downloaded from turnitinuk.com for distribution to other markers outside Turnitin after the Post Date. The window of opportunity is negotiated by course leaders, markers and intermediaries.
| Top of page |
Definition of anonymous marking for online assessment
After considering a range of possible definitions, the Board of Examiners has settled on the following definition of anonymous marking on Moodle, Turnitin or other online environment: student names are absent from their submissions at the point of marking.
...
There is no expectation that student work remains permanently anonymous to a marker after they have finished marking.
| Top of page |
What proxy identifier should be used instead of the student name?
Why is a proxy identifier needed at all? Because staff, sometimes working offline with downloaded files, need a robust, convenient way to refer to any given student, reconcile their markings with those of other markers, and update the student's record in the absence of a name.
...
| Info |
|---|
Unsure whether to use the SRN? Some E-Learning Champions have commented that the SRN is longer and therefore easier for students to mistype, and brings the extra workload of an intermediary stage to match it with the Candidate Number used in Portico. |
| Top of page |
Open second marking
Guidance on marking in Turnitin can be found in UCL miniguide M20 sub-guide (d).
Give or take some variation in settings, the inbox will look something like this (click for full size):
Full open second marking
Since second markers need to view the markings of first markers, the order of marking is important.
...
Where individual markers are allocated marking based on subject specialism, students may be instructed to include a very short identifying phrase – perhaps the question number - in the submission filename and submission title, so that markers can view the contents of the inbox and identify the work they need to mark.
| Top of page |
Sampled open second marking
Where Grademark has been used and second marking precedes the Post Date, the submission inbox allows the second marker to apply the sampling criteria by applying the sort function the numeric mark column so as to easily find firsts, fails, upper borderlines, etc. In addition their attention could be directed to any particular submissions with reference to the PaperID.
...
Please note that Digital Education is unable to offer technical support for your turnitinuk.com activity - Turnitin does offer support though.
| Top of page |
Open second marking after the Post Date
Currently in Turnitin, the Post Date permanently removes student anonymity.
...
Please note that Digital Education is unable to offer technical support for your turnitinuk.com activity - Turnitin do offer their own support though.
| Top of page |
Blind second marking
Full blind second marking
Since the order of marking is irrelevant here, markers can take advantage of the online environment by both accessing the work they need to mark at the same time; the first need not wait for the second and both can use the entire marking period. However, they cannot use Grademark, since their markings would be visible to each other i.e. they need to record marks and feedback outside Turnitin.
It is now possible to bulk-download the Turnitin inbox of anonymised work, so one marker can give feedback via Turnitin, while the other looks at the work outside Turnitin.
| Top of page |
Sampled blind second marking
The guidance is similar to that for 'Full blind second marking', except that it is on selected submissions. So markers would either need to be instructed about how to sort the inbox to take the sample, or an intermediary would need to download and re-anonymise the original submissions and send these securely to the blind second marker outside Turnitin, along with any marksheets required.
| Top of page |
Blind second marking after the Post Date
See the section on 'Open second marking after the Post Date' – the difference would be that the intermediary would pass anonymised original submissions (not the marked ones) to the second marker with a blank marksheet (not a completed one).
| Top of page |
Moderation
The moderator is tasked with checking the assignation of marks on a full set of submissions. If Grademark has been used and moderation happens before the Post Date, moderators can view the contents of the submission inbox on Turnitin, including the markings. If markings have been made outside Turnitin, then these need to be sent to the moderator either by the markers or via an intermediary.
| Top of page |
Sampled moderation
External examiners sometimes undertake sampled moderation. Unless a window of opportunity could be made before the Post Date lifts anonymity, an intermediary would need to pass the examiner(s) a downloaded, re-anonymised sample of marked work as set out in 'Open second marking after the Post Date' or 'Blind second marking after the Post Date', above.
| Top of page |
Moderation after the Post Date
The guidance here is the same as for 'Open second marking after the Post Date'.
| Top of page |
External examining
Since external examiners are routinely provided with an UCL User ID, they could be given a role in a Moodle course area which allows them access to its respective submission inboxes.
| Top of page |
Sampled external examining
The guidance here is the same as for 'Open second marking after the Post Date'. The submission inbox allows sorting by various columns including numeric mark, allowing examiners to easily find firsts, fails, upper borderlines, and also allows submissions to be randomly selected. In addition their attention could be directed to particular submissions with reference to the PaperID.
| Top of page |
External examining after the Post Date
The guidance here is the same as for 'Open second marking after the Post Date'.
| Top of page |
Questions and Answers
Isn't this too complicated to offer any advantages over paper processes?
...
| Info | ||
|---|---|---|
| ||
|
| Top of page |
Footnotes
* Currently draft so awaiting link.
...