On this page:
Introduction
The guidance below takes you through the several different anonymous marking scenarios with Moodle Assignments afforded by UCL's Marking Policy.
See Section 4.4 'Anonymity' Marking and Moderation Regulations of the Academic Manual.
This guidance is bound to evolve in line with policy, technologies and ideas from departments. You are invited to contact Digi Ed with any questions or suggestions - Digi Ed will amend this guidance accordingly and keep the E-Learning Champions informed of important improvements as they arise.
| Top of page |
Terms
- Anonymous marking: see below for a definition and explanation.
- Bulk-download: downloading the contents of a submission inbox for marking outside of Moodle.
- Candidate number: a unique identifier which students receive annually.
- Combinations of blind/open, full/sampled second marking: refer to the UCL Marking Policy*.
- External examining: refer to the UCL Marking Policy.
- Grader: Moodle's own marking environment.
- Identifier: the unique number that Moodle gives to each anonymised student (n.b. this doesn't correspond to the Student Number or Candidate Number)
- Intermediary: somebody without marking responsibilities, perhaps in an administrative or technical role.
- Markings: all notes, specific and general comments, and numeric marks made by markers.
- Moderating: refer to the UCL Marking Policy.
- Proxy identifier: see below for a definition and explanation.
- Student Record Number (SRN): this appears on the front of every student's ID card.
- Submission filename: the name of the file(s) submitted; this becomes important in cases where students' work is downloaded for marking outside of Turnitin.
- Submission title: the title students type into Turnitin when submitting their work. This displays in the Turnitin inbox and can be used to allocate submissions to subject specialist markers.
| Top of page |
Definition of anonymous marking for online assessment
After considering a range of possible definitions, the Board of Examiners has settled on the following definition of anonymous marking on Moodle, Turnitin or other online environment:
Student names are absent from their submissions at the point of marking.
Instead of their name, students are instructed to use a unique proxy identifier:
There is no expectation that student work remains permanently anonymous to a marker after they have finished marking.
| Top of page |
What proxy identifier should be used instead of the student name?
Why is a proxy identifier needed at all? Because staff, sometimes working offline with downloaded files, need a robust, convenient way to refer to any given student, reconcile their marks with those of other markers, and update the student's record in the absence of a name.
Until a technical solution becomes available, students should be instructed to include this proxy identifier in both the title and file name of their submission.
The recommended proxy identifier is the Candidate Number. The Student Record Number (SRN) may be used instead, if preferred.
The proxy identifier isn't generated automatically, so students need to be instructed to title their submission and file name so as to include it.
Unsure whether to use the Student Record Number? Some E-Learning Champions have commented that the Student Record Number is longer and therefore easier for students to mistype, and brings the extra workload of an intermediary stage to match it with the Candidate Number used in Portico.
| Top of page |
Open second marking
Full open second marking
Where? First and second marking can take place either online or offline. First marking online may make things easier for two reasons. Feedback will be ready for release to students, and the Second Marker will be able to log in directly rather than having to cross-reference between emailed or shared files and Moodle.
Feedback. If the submitted work is in PDF format markers can directly annotate online; if more than one marker is doing this then each needs to distinguish between different markers' notes by typing initials (e.g. M1 or M2) into their own comments i.e. to fulfil record keeping requirements. However, if First Marker feedback is the only feedback students are expected to see, then the Second Marker needs to make their comments outside of Moodle. Bulk downloading the inbox along with a Grading Worksheet is a good method. If the submitted work is word processed or other file format, it can be bulk downloaded for annotation using e.g. MS Word Comments, and bulk uploaded as feedback files (caveat - only one marker should carry out a bulk upload. The reason is that a second upload would overwrite the first because the file names are necessarily identical).
Grades. It may best fulfil record keeping requirements to use the Moodle Assignment's Grading Worksheet which can be downloaded and completed by the First Marker and passed to the Second Marker, who may complete their own separate Grading Worksheet. These can be manually compared when the time comes to reconcile the grades, and a single finalised version uploaded to Moodle. Alternatively, this process can be carried out online directly in the Moodle Grader but be aware that when the feedback is released, students will see all marks or comments there, so it may need to be tidied, which poses obstacles to record keeping for the examinations board, external examiners &tc.
Records may take the form of the Grading Worksheet, feedback comments on Moodle, and files of annotated student work (separate for First and Second Marker).
| Top of page |
Sampled open second marking
As above, but with a sample defined by local policy.
To obtain the sample, the Moodle Assignment inbox can be sorted by grade to easily find Firsts, Fails and borderlines and access the submissions. This may be the most convenient approach.
If an offline Grading Worksheet has been used, this can also be sorted using e.g. MS Excel, but finding the corresponding student's work may be less convenient since the proxy identifier on the worksheet and in the filename of the submission is a longish number.
| Top of page |
Open second marking after releasing first marker feedback
This allows a quick return of feedback to students.
Before the First Marker feedback is released (which necessarily lifts student anonymity) the Second Marker downloads the anonymous inbox and any feedback. The Second Marker can refer to the First Marker's markings and carry out Second Marking outside of Moodle. If students are to receive Second Marker comments (as well as First Marker comments), these need to be uploaded individually as 'separate' feedback files (i.e. they CANNOT both be bulk-uploaded since the original filename acts as a unique identifier, and is required to match files with students. A subsequent bulk upload risks overwriting First Marker feedback.
Students should be made aware that the First Marker's marks are provisional and therefore subject to change.
| Top of page |
Blind second marking or independent marking
Full blind second marking
Second marking takes place outside of Moodle. Before First Marker marking begins, the Second Marker bulk downloads the inbox and Grading Worksheet.
Feedback can be returned to students, through bulk uploading the marked submissions and the completed Grading Worksheet.
| Top of page |
Sampled blind second marking
First marking can take place in Moodle, while second marking takes place outside of Moodle. Before First Marker marking begins, the Second Marker bulk downloads the inbox and identifies the sample (either by eye or by carefully filtering the data in the Grading Worksheet) according to the agreed policy.
First and Second Markers reconcile the marks and keep records of both sets of marks and feedback.
| Top of page |
Blind second marking after releasing first marker feedback
First marking can take place in Moodle, while second marking takes place outside of Moodle. Before first marking begins, the Second Marker bulk downloads the inbox of submissions along with a Grading Worksheet.
After first marking - which may take place in Moodle - is complete, this feedback is released to students. Students should be made aware that the First Marker's marks are provisional and therefore subject to change.
Meanwhile second marking is carried out. First and Second Markers reconcile the marks. If students are to receive Second Marker comments, these need to be uploaded individually as separate feedback files (i.e. they cannot be bulk-uploaded since the original filename acts as a unique identifier, required to match files with students, and so risks overwriting First Marker feedback which may have been bulk uploaded.
| Top of page |
Bulk downloading submissions and uploading feedback files
Select 'Download All Submissions' within the assignment to to bulk download.
Select Upload Multiple Feedback Files in a zip to upload feedback files. You can then move your feedback files into the associated student folder.
Moderation or Check Marking
If first marking is completed in Moodle, then before the marks are released to students (i.e. while student anonymity is still in place), moderation or checking is carried out.
| Top of page |
Sampled moderation or Check Marking
As above, but filter the inbox by grade to find the sample you need.
| Top of page |
Moderation or Check Marking after releasing first marker feedback
First marking is completed on a bulk downloaded inbox, which allows a still-anonymous marked work to be passed to checkers or moderators outside of Moodle. Meanwhile it can be bulk-uploaded for students to access their marks and feedback on Moodle.
| Top of page |
External examining
See below.
| Top of page |
Sampled external examining
See below.
| Top of page |
External examining after marks have been released
This is the usual situation. Students need to remain anonymous, and marking and how marks are arrived at must be documented and transparent for external examiners (as well as examination boards, students, and if necessary, grievance panels).
External Examiners can be given the marked submissions and 'completed' Grading Worksheet. Note that this can't be downloaded from Moodle, but already exists with the marker(s). It's probably a good idea to work out a departmental system for keeping these, perhaps a shared network drive since this will be both secure and backed up.
| Top of page |
Questions and Answers
See our Guidance on anonymous marking in Turnitin since there is considerable overlap between Turnitin and Moodle with respect to the questions.
Do word-processed documents include the author names in the properties?
Microsoft documents may include personal or hidden data. Of necessity, the responsibility to provide an anonymised document must fall to each student - see Microsoft's guidance on removing personal or hidden data in documents. The short answer is, anonymity means students names are removed from the script at the point of marking. The personal data isn't blatantly obvious, so if you don't look for it you won't be confronted with it.
| Top of page |
Is it possible to hide the grader's name from the student?
This is a feature of Moodle. In the settings of a Moodle assignment, under Grade you can set Hide grader identity from students to yes to hide the name of the grader to the student. This can avoid student confusion, especially when multiple markers and administrators handle assignment submissions.
| Top of page |
Guidance on anonymous marking in Moodle - including second marking, moderating and external examining by the Digital Education team is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at https://wiki.ucl.ac.uk/display/MoodleResourceCentre/.