Contents
| Anchoranchor | |||||
|---|---|---|---|---|---|
|
On this page:
| Table of Contents |
|---|
...
...
Introduction
The UCL Board of Examiners of 26th February 2014 resolved '[t]hat a flexible and simple framework should be drafted which sought to prevent allegations of bias and outlined clear definitions of acceptable standards and good practice in this area...'
Approved by at the July 2014 Board of Examiners meeting, the guidance below takes you through the several different anonymous marking scenarios with Turnitin afforded by UCL's draft Marking Policy. It is concerned with planning and as such is at quite a high level; elsewhere ELE provides instructions about the particulars of setting up and operating Turnitin.
...
| Info |
|---|
This guidance is bound to evolve in line with policy, technologies and ideas from departments. You are invited to contact ELE with any questions or suggestions - ELE will keep the E-Learning Champions informed of important changes as they arise. |
| Top of page |
Terms
- Anonymous marking
- Bulk-download: downloading the contents of a submission inbox for marking outside Turnitin.
- Combinations of blind/open, full/sampled second marking: refer to the UCL Marking Policy Currently draft.*.
- External examining: refer to the UCL Marking Policy.
- Grademark: Turnitin's own marking environment.
- Intermediary: somebody without marking responsibilities, perhaps in an administrative or technical role.
- Markings: all notes, specific and general comments, and numeric marks made by markers.
- Moderating: refer to the UCL Marking Policy.
- Moodle Assignment: Moodle offers a separate online assessment environment. Its marking and feedback possibilities currently differ from Turnitin's. Only Turnitin offers an Originality Report, and Turnitin's suite of online annotation tools is slightly more developed.
- Paper ID: a numeric identifier unique to each paper, generated Turnitin, but currently irrelevant outside Turnitin.
- Post Date: point at which marks and feedback are released, necessitating the lifting of student anonymity.
- Submission filename: the name of the file(s) submitted; this becomes important in cases where students' work is downloaded for marking outside Turnitin.
- Submission title: the title students type into Turnitin when submitting their work. This displays in the Turnitin inbox and can be used to allocate submissions to subject specialist markers.
- submit.ac.uk: allows access (where there are existing permissions) to a Turnitin submission inbox independently of Moodle; affords downloading of Grademarked work and - though only after the Post Date - original unmarked work.
- Window of opportunity: a phrase used in this guidance to refer to a period just after Grademarking (if used) is complete and just before the Post Date de-anonymises students. During this window the still-anonymous, Grademarked submissions, complete with markings, can be downloaded from submit.ac.uk for distribution to other markers outside Turnitin after the Post Date. The window of opportunity is negotiated by course leaders, markers and intermediaries.
...
| Top of page |
Definition of anonymous marking for online assessment
After considering a range of possible definitions, the Board of Examiners has settled on the following definition of anonymous marking on Moodle, Turnitin or other online environment: student names are absent from their submissions at the point of marking. Instead of each student name, students are instructed to use a unique proxy identifier.
What proxy identifier should be used instead of the student name?
Multiple Why is a proxy identifier needed? Because multiple markers, sometimes working offline with downloaded files, need a convenient proxy identifier in order way to refer to the same student and to ultimately be able to reconcile their various markings and contribute to the student's record in the absence of a name.
Until we arrive at a technical solution becomes available, students should be instructed to include this proxy identifier in both the title and filename file name of their submission_._
The recommended proxy identifier can be is the Candidate Number or the . The Student Record Number , depending on policy in departments.
...
Since second markers need to view the markings of first markers, order is important(SRN) may be used instead.
The proxy isn't included automatically, so students need be instructed to include it (more on this below).
| Info |
|---|
Unsure which to use? Some E-Learning Champions have commented that the SRN it is less anonymous, longer and therefore easier for students to mistype, and requires an intermediary stage to match it with the Candidate Number used in Portico. |
Open second marking
Full open second marking
Since second markers need to view the markings of first markers, order is important. First markers access the submission inbox and marks with Grademark according to departmental guidelines. At an agreed date and time (before the Post Date), the second marker accesses the same inbox. Depending on local policy, second markers can use Grademark to add their markings alongside the first marker's. Turnitin will not currently distinguish the two sets of comments, so the second marker can initial or otherwise identify theirs.
...
Where individual markers are allocated marking based on subject specialism, students may be instructed to include a very short identifying phrase – perhaps the question number - in the submission filename and submission title, so that markers can view the contents of the inbox and identify the work they need to mark.
...
...
Sampled open second marking
Where Grademark has been used and second marking precedes the Post Date, the submission inbox allows the second marker to apply the sampling criteria by applying the sort function the numeric mark column so as to easily find firsts, fails, upper borderlines, etc. In addition their attention could be directed to any particular submissions with reference to the PaperID.
Where Grademark has been used and second marking happens after the Post Date, an intermediary accesses the inbox via submit.ac.uk For instructions see https://wiki.ucl.ac.uk/display/MoodleResourceCentre/Staff+Turnitin+FAQs#StaffTurnitinFAQs-HowdoIbulkdownloadTurnitinassignments%3F. Note that submit.ac.uk only allows Grademarked work to be downloaded before the Post Date i.e. not original unmarked work. and sorts it to identify a sample which is then downloaded and securely passed, outside Turnitin, to the second marker. If a brief window of opportunity can be made to download this sample after marking is complete but before the Post Date de-anonymises student work, this both potentially saves a great deal of time for whomever carries out this task and enables a marker to do it. Otherwise, an intermediary needs do it, manually re-anonymising all submissions by removing the student name Turnitin appends after the Post Date.
...
Open second marking after the Post Date
Currently in Turnitin, the Post Date permanently removes student anonymity.
...
Before the Post Date submit.ac.uk allows staff to bulk-download only submissions which have been Grademarked submissions , with anonymity intact. Only after the Post Date can original files be downloaded – but after the Post Date , each filename will be prefixed Turnitin prefixes each file name with the student's name, and so , thus compromising anonymity. In this case, an intermediary would manually need to re-anonymise the submissions by removing these names (a considerable task).
Moodle Assignment, on the other hand, allows bulk-download of anonymous work at any time until the marks are released to students.
Do markers need to be excluded from a Moodle area containing an assignment after that assignment's Post Date?Without reference to work and markings, markers are likely to experience more obstacles to discussing the feedback with individual students, and may be hampered from considering students' development over time. According to the definition of anonymous marking in use here, only markers who will be marking after the Post Date would need to be excluded from a Moodle area containing an assignment (because students names will have been revealed by the time those markers access the submission inbox). Otherwise markers may remain in their usual roles in those Moodle areasfrom a Moodle area containing an assignment after that assignment's Post Date?
No, because without reference to work and markings, markers are likely to experience more obstacles to discussing the feedback with individual students, and may be hampered from considering students' development over time. According to the definition of anonymous marking in use here, only markers who will be marking after the Post Date would need to be excluded from a Moodle area containing an assignment (because students names will have been revealed by the time those markers access the submission inbox). Otherwise markers may remain in their usual roles in those Moodle areas.
Doesn't reuse of the Candidate Number compromise anonymous marking?
Technical limitations oblige us to balance competing goods of anonymous marking and assessment feedback. A more defensive view of anonymity would seek a technical solution which treated assessors as if they were deliberately intending discrimination, rather than simply subject to unconscious bias. Since there is a grievance process for students who wish to allege intentional discrimination, it is considered sufficient here to take measures to disrupt unconscious bias. There is a recognition on the part of the Board of Examiners that the Candidate Number may be used on multiple occasions over the year, and that (because of technicalities of updating each student's record) its repeated use may be the only feasible means of communicating assessment feedback to students in circumstances of anonymous marking.
Doesn't Tutor access to the Participation Report compromise anonymous marking?
...
ELE recommend that the Post Date is the same as a the date on which students have been told to expect feedback. If anonymous marking needs to happen after that, the original unmarked submissions can be downloaded by an intermediary, re-anonymised and sent to the marker. Students can be apprised that their mark is provisional, and Moodle's Turnitin Integration allows marks to be changed after the Post Date.
* No link at this stage - will supply when the policy is approved and published.