...
| Info |
|---|
This guidance is bound to evolve in line with policy, technologies and ideas from departments. You are invited to contact ELE with any questions or suggestions - ELE will keep the E-Learning Champions informed of important changes as they arise. |
| Top of page |
Terms
- Anonymous marking: see below.
- Bulk-download: downloading the contents of a submission inbox for marking outside Turnitin.
- Combinations of blind/open, full/sampled second marking: refer to the UCL Marking Policy Currently draft*.
- External examining: refer to the UCL Marking Policy.
- Grademark: Turnitin's own marking environment.
- Intermediary: somebody without marking responsibilities, perhaps in an administrative or technical role.
- Markings: all notes, specific and general comments, and numeric marks made by markers.
- Moderating: refer to the UCL Marking Policy.
- Moodle Assignment: Moodle offers a separate online assessment environment. Its marking and feedback possibilities currently differ from Turnitin's. Only Turnitin offers an Originality Report, and Turnitin's suite of online annotation tools is slightly more developed.
- Paper ID: a numeric identifier unique to each paper, generated Turnitin, but currently irrelevant outside Turnitin.
- Post Date: point at which marks and feedback are released, necessitating the lifting of student anonymity.
- Submission filename: the name of the file(s) submitted; this becomes important in cases where students' work is downloaded for marking outside Turnitin.
- Submission title: the title students type into Turnitin when submitting their work. This displays in the Turnitin inbox and can be used to allocate submissions to subject specialist markers.
- submit.ac.uk: allows access (where there are existing permissions) to a Turnitin submission inbox independently of Moodle; affords downloading of Grademarked work and - though only after the Post Date - original unmarked work.
- Window of opportunity: a phrase used in this guidance to refer to a period just after Grademarking (if used) is complete and just before the Post Date de-anonymises students. During this window the still-anonymous, Grademarked submissions, complete with markings, can be downloaded from submit.ac.uk for distribution to other markers outside Turnitin after the Post Date. The window of opportunity is negotiated by course leaders, markers and intermediaries.
...
After considering a range of possible definitions, the Board of Examiners has settled on the following definition of anonymous marking on Moodle, Turnitin or other online environment: student names are absent from their submissions at the point of marking. Instead of each student name, students are instructed to use a unique proxy identifier.
| Top of page |
What proxy identifier should be used instead of the student name?
...
| Info |
|---|
Unsure which to use? Some E-Learning Champions have commented that the SRN it is less anonymous, longer and therefore easier for students to mistype, and requires an intermediary stage to match it with the Candidate Number used in Portico. |
| Top of page |
Open second marking
Full open second marking
...
Where individual markers are allocated marking based on subject specialism, students may be instructed to include a very short identifying phrase – perhaps the question number - in the submission filename and submission title, so that markers can view the contents of the inbox and identify the work they need to mark.
| Top of page |
Sampled open second marking
...
Where Grademark has been used and second marking happens after the Post Date, an intermediary accesses the inbox via submit.ac.uk For instructions see https://wiki.ucl.ac.uk/display/MoodleResourceCentre/Staff+Turnitin+FAQs#StaffTurnitinFAQs-HowdoIbulkdownloadTurnitinassignments%3F. Note that submit.ac.uk only allows Grademarked work to be downloaded before the Post Date i.e. not original unmarked work. and sorts it to identify a sample which is then downloaded and securely passed, outside Turnitin, to the second marker. If a brief window of opportunity can be made to download this sample after marking is complete but before the Post Date de-anonymises student work, this both potentially saves a great deal of time for whomever carries out this task and enables a marker to do it. Otherwise, an intermediary needs do it, manually re-anonymising all submissions by removing the student name Turnitin appends after the Post Date.
| Top of page |
Open second marking after the Post Date
...
Where marking has taken place outside Turnitin, the process is more convoluted. From submit.ac.uk an intermediary downloads the original files (only available after the Post Date), re-anonymises these by removing the name Turnitin appends to the submission filename, ensures the Candidate Number is present in the filename, and securely passes these to the second marker along with the first marker's markings.
...
| Top of page |
Blind second marking
...
Full blind second marking
Since the order of marking is irrelevant here, markers can take advantage of the online environment by both accessing the work they need to mark at the same time; the first need not wait for the second and both can use the entire marking period. However, they cannot use Grademark, since their markings would be visible to each other.
The following alternative approaches are available:
- Moodle Assignment may offer advantages over Turnitin for blind marking since it would allow staff to download their own copies of the anonymous submissions, a single marker could bulk-upload a file of agreed numeric marks, summary comments and files for students containing feedback. The student feedback could include direct annotations using a word-processor or PDF annotation software Providing its use does not conflict with local policy on using Turnitin's Originality Check.. It would also be feasible for the . It would also be feasible for the first marker to mark within Moodle while the second marker received the submissions outside Moodle.
- If Or, if Turnitin is preferred, it could be used to view submissions but markers should not use Grademark. Instead they could type feedback onto marksheets provided outside Turnitin.
- Where Or, if Turnitin is preferred and each marker is expected to Grademark the work, this would involve setting up a duplicate submission for each marker. There are some important caveats here: avoid using Moodle's Duplicate function but create entirely fresh, separate Turnitin assignments; only one assignment should be set to contribute the work to Turnitin's repository To avoid 100% match on the Originality Report.; students would need to be carefully instructed to submit to both; and the external examiner should be given all marked copies.
...
| Top of page |
Sampled blind second marking
[To come]
...
The guidance is similar to that for full blind second marking, except that markers would either need to be instructed as to how to sort the inbox to take a sample, or an intermediary would need to download and re-anonymise the original submissions and send these securely to the blind second marker outside Turnitin.
| Top of page |
Blind second marking after the Post Date
See the section on 'Open second marking after the Post Date' – the difference would be that the intermediary would pass anonymised original submissions (not the marked ones) to the second marker with a blank marksheet (not a completed one).
...
| Top of page |
Moderation
The moderator is tasked with checking the assignation of marks on a full set of submissions. If Grademark has been used and moderation happens before the Post Date, moderators can view the contents of the submission inbox on Turnitin, including the markings. If markings have been made outside Turnitin, then these need to be sent to the moderator either by the markers or via an intermediary.
...
| Top of page |
Sampled moderation
External examiners sometimes undertake sampled moderation. Unless a window of opportunity could be made before the Post Date, an intermediary would need to pass the examiner(s) a downloaded, re-anonymised sample of marked work as set out in 'Open second marking after the Post Date' or 'Blind second marking after the Post Date', above.
...
| Top of page |
Moderation after the Post Date
The guidance here is the same as for 'Open second marking after the Post Date'.
...
| Top of page |
External examining
Since external examiners are routinely provided with an UCL User ID, they could be given a role in a Moodle course area which allows them access to its respective submission inboxes.
...
| Top of page |
Sampled external examining
The guidance here is the same as for 'Open second marking after the Post Date'. The submission inbox allows sorting by various columns including numeric mark, allowing examiners to easily find firsts, fails, upper borderlines, and also allows submissions to be randomly selected. In addition their attention could be directed to particular submissions with reference to the PaperID.
...
| Top of page |
External examining after the Post Date
The guidance here is the same as for 'Open second marking after the Post Date'.
...
| Top of page |
Questions and Answers
Isn't this too complicated to offer any advantages over paper processes?
...
ELE recommend that the Post Date is the same as a the date on which students have been told to expect feedback. If anonymous marking needs to happen after that, the original unmarked submissions can be downloaded by an intermediary, re-anonymised and sent to the marker. Students can be apprised that their mark is provisional, and Moodle's Turnitin Integration allows marks to be changed after the Post Date.
| Top of page |
Footnotes
* No link at this stage - will supply when the policy is approved and published.
| Top of page |