Marking and Moderation Variations and Workflows
This page describes some of the different marking workflows that can be accommodated in WISEflow. It is strongly recommended that you read this guidance to aid you in planning the marking and moderation phase of your assessments.
At UCL all modules must be second marked in some way, in accordance with the regulations set out in the Academic Manual. This helps ensure that the final grades awarded are accurate and appropriate.
Access the Feedback and Assessment Sharepoint site to read the Second Marking Overview & Myth Busters.
Marking Variations at UCL
It is possible to combine the marking variations listed below (e.g. your assessment can have blind full second marking; sample, checked second marking; etc.)
Variation | Description |
|---|---|
Full Second Marking | Second markers mark or check all assessments |
Sample Second Marking | Second markers mark or check a sample, based on defined criteria, of the full set of assessments |
Independent | Each marker enters and submits a mark. The two marks are subsequently reconciled to agree the mark for the assessment |
Checked | The second marker determines whether the mark awarded by the first marker is appropriate, but does not give a separate mark. The second marker confirms the mark if appropriate, and brings it to the attention of the first marker if not. Usually only appropriate for assessments where answers can be scored objectively rather than requiring qualitative judgement on the part of the markers (e.g. assessments featuring MCQs) |
Blind | The second marker is not informed of the first marker’s marks and/ or comments |
Open | The second marker is informed of the first marker’s marks and comments before commencing and can take these into account |
Which Marking Variation should I use?
Dissertations and research projects: these must use full, independent second marking as per the regulations outlined in the Academic Manual.
Exams and assessments comprised of MCQs or other quantitative exercises: sample, checked second marking is recommended.
All other assessment types: rather than making blanket recommendations, we encourage you to consider your existing practice and broader practice within your department or faculty.
Once you have decided how the work is going to be marked, you should then determine which marking workflow will be most suitable to handle the marking process in WISEflow and communicate this to the person or team responsible for setting up the assessment (flow Manager).
Marking Workflows in WISEflow
IMPORTANT: Aim to set up the marking workflow, marker allocation and any rubrics that will be used in plenty of time before the start of the marking period. Marking decisions that are made at the last minute or very late during the marking process are likely to cause errors and complications, and should be avoided wherever possible.
Comparison Table
| Standard allocation | Item-based marking | Rubric-based marking |
|---|---|---|---|
Description | Markers are allocated to students | Markers are allocated to specific items/ questions | Markers are allocated to students and are allocated a rubric |
Available flow types | All flow types | FLOWmulti | All flow types |
How are grades finalised | By Marker agreement or Reviewer with special permissions | By Manager | By Manager |
Feedback options (optional unless stated) |
|
|
|
Supported marking variations |
|
|
|
Notes |
|
|
|
Standard Allocation
Standard allocation is the practice of allocating markers to students (participants) in WISEflow. Markers will see the entire student submission and assess it based on a variety of marking criteria. You will probably use this marking workflow for the majority of your assessments. Markers can be allocated to a percentage of the cohort or to specific students and multiple markers can be allocated to the same student(s).
It is possible to attach a rubric/ grading form to a flow using the standard allocation workflow but not essential.
Related Guidance:
Item-Based Marking (FLOWmulti only)
Rather than allocating a marker to entire student submissions or exam scripts, item-based marking allows markers to be allocated to a specific part of the assessment. It can be useful in scenarios where:
Markers need to mark across submissions (e.g. all of question 1, then all of question 2) rather than the traditional linear approach where complete papers are marked in turn.
Markers with different specialisms need to assess different parts of the exam.
It should generally not be used if the exam features optional (“X out of Y”) questions.
Item-based marking can be used in assessments which feature a combination of auto-scored and manually marked questions, or where all questions require manual marking. Markers will complete manual marking for their allocated sections of the exam and their submitted scores, along with any auto-scored questions, will be collated and converted into grades by a user in the Managing role.
Note: Think carefully about how you will structure your exam paper if you choose this marking workflow. Ideally, each item should contain only questions that a specific marker needs to assess (i.e. try not to combine auto-scored and manually marked questions in one item, or questions that multiple different markers need to score).
This workflow is exclusive to FLOWmulti type flows and is not compatible with the use of rubrics. Item-based marking can be set up by a user with Manager access to the assessment.
Related Guidance:
Rubric-Based Marking
Note: Rubric-based marking is a specific, advanced marking workflow. It is different to just attaching a rubric to a flow and using a standard marker allocation.
Rubric-based marking allows markers to assess student submissions based on specific criteria. They will be able to view the entire submission, but may only be marking part of it. It can be useful in scenarios where:
Markers with different specialisms need to assess different parts of the submission.
The assessment features X out of Y questions (e.g. “Answer THREE of the following FIVE questions”).
Final grading decisions need to be automated based on contributions from several markers.
Markers will complete their allocated rubric for each submission and their submitted scores will be collated and converted into grades by a user in the Managing role.
This workflow can be used on all flow types, but cannot be enabled on FLOWmulti type flows if item-based marking is already in use. Rubric-based marking can be set up by a user with Manager access to the assessment. Be aware that:
All rubrics that have been added to the flow must be used to calculate the final grade (e.g. if Rubric A and Rubric B have been added to the flow, scores from both will be required. You cannot use only Rubric A to mark some students and only Rubric B to mark others)
The same rubric can be allocated to multiple markers (e.g. Markers 1 and 2 will use Rubric A, Markers 3 and 4 will use Rubric B)
You cannot allocate multiple rubrics to a marker (e.g. Marker 1 cannot use both Rubric A and Rubric B; in this scenario a potential solution could be creating Rubric C, which contains all of the components that Marker 1 needs to use)
Markers must be allocated to the students they are going to mark (e.g. Marker 1 is using Rubric A to mark students α, β and γ. To help distribute the workload, Marker 2 will also use Rubric A to mark students δ, ε and ζ)
Related Guidance:
Tip: If none of these workflows seem like they quite fit your envisioned marking process, you can always contact the Digital Assessment Team at assessment-advisory@ucl.ac.uk for advice or additional insight.