Rubrics for X of Y/ Optional Questions
This guidance covers the creation of rubrics to address scenarios where students are asked to select which questions to answer (e.g. “Answer THREE of the following FIVE questions”) and assumes you are somewhat familiar with how to create a rubric in WISEflow. For more general guidance on creating rubrics, see Creating a rubric.
Key Considerations
You will probably need to set up a custom rubric for assessments featuring optional questions. Note that if offline marking is important, only multiple choice and range scoring question types are compatible with WISEflow’s offline marking functionality (if written commentary is needed, this could still be added via a comment using submission feedback, i.e. separate from the rubric).
Your rubric will need to be flexible enough to address a variety of outcomes, favour a more generic/ open design rather than specific. Fill in the blanks drop down questions can be very useful for this.
If using rubric-based marking: final grades are dependent on rubric scores (i.e. rubrics cannot be totally holistic).
If the assessment contains auto-scored questions (FLOWmulti only): there is no link between automatic scoring and scores awarded via a rubric - you will need to make sure there is a place in your rubric where the marker can confirm the automated score.
X of Y and Rubric-based Marking
In this workflow, each marker uses a rubric to assess part of the student submission. The students must be scored using every rubric that is attached to the flow (i.e. you cannot just use a rubric some of the time for a subset of the cohort).
Example scenario: an exam contains 12 questions, students choose 10 of them to answer. The marking team is set up so that:
One marker is marking questions 1 - 7
One marker is marking questions 8 and 9
One marker is marking questions 10 - 12
On the surface, an obvious solution is to create one rubric for each marker (rubrics A, B and C):
However, if several students choose not to answer questions 8 or 9, rubric B wouldn’t be needed - but because all rubrics have to be used in rubric-based marking, WISEflow would still expect a marker to complete rubric B and score these students. This creates a lot of unneccesary extra work and may also lead to inaccurate final grade calculations!
Instead: creating a single, generic rubric which all markers can use will allow us to avoid any extra workload and ensure that the overall grades are calculated correctly:
X of Y and Standard Marker Allocation
In this workflow, markers assess the student submission in its entirety. Rubrics can be more holistic if needed (i.e. unlike rubric-based marking, there is not an absolute requirement for precise scoring) and will be used to deliver the overall grade rather than a smaller score that forms part of the overall grade.
Assessments or Exams including Auto-scored Questions (FLOWmulti only)
There is no link in WISEflow between FLOWmulti’s scoring feature and rubrics, so if questions with automatic scoring are also part of an assessment with X of Y questions, it will be important to factor a way to record this into the rubric you design.
Generally, a simple range scoring question should suffice for this:
There is likely to be additional complexity to consider where an exam with auto-scored and X of Y questions also requires a rubric-based marking workflow.
Including mark breakdown for sub-questions
If a question is comprised of several sub-questions (e.g. Question 1a), 1b), etc.), it may be helpful to reflect this when recording the marks awarded. In order to ensure flexbility in your X of Y rubric, this should only ever be a zero-weighted component and should not contribute to the overall grade.
Instead, you can add a range scoring question directly underneath the area for the breakdown of marks and record the total score as shown in the example below:
Recommended Question Types
Tip: There are already some templates for X of Y questions available in the rubrics content bank! For steps on how to include these in your rubric, see Rubric templates.
Important: If a custom rubric contains any question type which is not supported for offline marking, you will not be able to export the rubric along with the grade sheet. This will mean that the rubric must be completed online/ in WISEflow.
Question Type | Example Use Case | Offline marking support |
|---|---|---|
Fill in the blanks: fill in text with drop down |
| No |
Fill in the blanks: fill in text |
| No |
Range scoring |
| Yes |
Multiple choice |
| Yes |
Essay |
| No |
Where you are using a question/ rubric component for cosmetic/ zero-weighted reasons (i.e. it should not contribute to the overall score or grade for the submission), you will need to make sure that you set the question score to 0 and tick the Practice question, do not score option under Scoring:
This ensures that completed rubrics don’t display “incorrect” answers when being viewed by a Reviewer (moderator, external examiner, etc.) or a student.
Once you have created your rubric, it is strongly recommended that you Preview it and check that all of the components work as you expect them to:
Handling optional questions can get really complex - not just in WISEflow but in many assessment platforms. If you’re still unsure of how to design your rubric or want additional support, contact your Faculty Learning Technology Lead or raise a ticket in MyServices.