This page outlines the required steps intended for colleagues involved in planning, delivering or marking departmental assessments on AssessmentUCL (WISEflow). These steps outline best practices and are designed to clarify the expectations of departments in delivering assessments on the platform, get the most out of the existing functionality and avoid some known pitfalls.  Like any educational technology, WISEflow has limits in terms of the range of requirements it can accommodate. The product owners (UNIwise) are working in close partnership with UCL to enable enhancements in response to feedback and use cases from departments.

Access further information about the roadmap for these developments.

The recommendations cover the following key areas, organised in stages of the assessment journey and are relevant to different users (depending on your role in assessments):

Additional resources:

• Guidance for staff

• Guidance for students

STEPS

Open	Open
Review the onboarding package for WISEflow to understand: the suitability of your departmental assessments responsibilities and requirements from you and colleagues in supporting and maintaining assessments on the platform the training resources available so that your staff are sufficiently prepared the level of support offered centrally to help you get started	Establish team capacity – training for all staff, carrying out platform tasks, admin support Gather key assessment data Agree your marking and feedback processes. Communicate early with staff and students	Create assessments via SITS integration and configure settings Create academic content online; i.e., MCQ exams, Rubrics etc. Attach assignment & supporting materials Ensure markers have been allocated to student submissions Add ECs and SORAs for students Signpost students to demo flows Activate assessment	Monitor student progress live via the platform Be aware of student support channels; ISD helpdesk, Assessment Query Form Add extra time for students (if needed); late EC requests etc. Schedule or send messages to students via the platform	Communicate with markers on deadlines and track progress. We recommend setting up a Teams site to do this, Inform markers of the administrator (Manager) who can resolve simple queries; i.e., access to assessment	Check for late submissions Notify students that feedback has been released Transfer full set of marks to Portico (and apply penalties/capping on Portico) Provide External Examiners with guidance View archived assessments (if needed)

1. ONBOARDING

Before proceeding with plans to run assessments on WISEflow, it is crucial that departments have a clear understanding of what it takes to run assessments smoothly on the platform. Departments should use the onboarding package below to ensure their suitability and readiness to transition their assessment and related processes to this platform.  The term 'onboarding' in this context refers to the process of moving individuals and processes successfully onto the Wiseflow platform.

i. Onboarding package

The onboarding package will allow you to determine if your assessments are suitable, understand the roles and responsibilities involved (including whether your staff resources and capacity are sufficient), and the training and support expectations from your department. Upon completing the onboarding package, should departments find the platform is compatible with their assessment needs, they are encouraged to arrange a consultation with a Digital Assessment Advisor for a more in-depth discussion of their plans.

Open

2. PLANNING 

Planning is arguably the most important stage when running assessments on WISEflow and will help avoid issues later in the process. Planning should be led by the Course or Module lead in collaboration with the Administrators, Markers and other team members as outlined below.

i. Establish capacity and availability of key staff. 

 To run successful assessments on the platform, your department needs to:

• Have a team in place, including administrators, markers, the course/module/programme lead and any other academics and/or externals. Administrative support is crucial as their role plays an essential part in ensuring assessments run smoothly from beginning to end. They are usually the central point of contact for your team and liaison for the Digital Assessment Team who provide advice and guidance.

• Ensure capacity of your team members to commit to training and carry out WISEflow related tasks -these commitments should be accounted for in your team’s workload model;

• Ensure your team understands their roles and responsibilities on WISEflow. These are determined by the designated roles on the platform (see What role might I have on WISEflow?)

• Ensure that the right people have the right roles on SITS so that they can be brought through using integration (see How are staff assigned in WISEflow?)

• Communicate with your team about expectations, timelines and so on and ensure that they have access to key assessment information and any relevant training guidance.

• Identify how and when to get support (see Support during an WISEflow assessment)

• Plan marking and feedback processes well in advance (see section directly below);

ii. Gather key assessment data

Following the preparatory meeting/communication, you will be asked to complete a spreadsheet with details such as; the dates for your assessment, marking period, who your assessors will be, and so on. These are required to set up the assessments on the platform.

iii.  Plan marking and feedback

To make marking and feedback run smoothly, we recommend that you plan by:

• Reviewing your marking processes. Are they overly complex? Is there any way to streamline workflows? Are your marking workflows suitable for WISEflow?  Please note: Currently there are no re-useable comments (e.g., Turnitin Quickmarks) on WISEflow but there are comment and annotation tools, as well as rubrics. Rubric marking on WISEflow can vary greatly from Turnitin, so it is important to check these features are compatible with your marking

• Agreeing on a shared marking approach which all markers should follow. Unlike Moodle or Turnitin, each marker in WISEflow has their own copy of a student submission which allows for first and second marking to be recorded separately. A nominated marker must then reconcile marks and enter a final agreed mark. Whilst this can seem like more work compared to Moodle or Turnitin, it ensures clearer records of double marking within the one platform. In future, we hope to automate aspects of the sampling and the reconciliation of marks.

• Ensuring markers have sufficient time to undertake training and ask them to schedule this in advance.

See also 5. Marking and Feedback

iv. Provide support and information to students

Following the setup of assessments on the platform, you should encourage students to complete the platform's demo assessments and advised to do a practice run within your teaching schedule, in advance of their first summative assessment. It is recommended that students complete the demo assessments to mitigate the likelihood of experiencing fewer issues during their live exams, such as leaving it too late to upload their submissions or browser/device issues that prevent accessing information and saving of answers. Therefore, we recommend:

• Familiarising yourself with student support guidance

• Communicating with students early about requirements and support available. A template for departments outlining essential information and guidance to students will be provided - departments should amend this template for their own use and circulate to their students.

• Include a practice and Q&A session in scheduled teaching that give students the opportunity to complete the standard demo flows and ask questions about their assessment. This is especially relevant if your assessments require complex scanning and/or upload requirements.

3. SETTING UP

i. Creating Assessments (Flows)

Once the key assessment data has been established, the assessments (referred to as “flows” in the platform) should be set up by the person with the Manager role. This is the Administrator and/or Exam Liaison Officer associated with the assessment(s). Please note academics are not allocated the Manager role.

Set up includes:

• creating the flow assessment instance and ensuring that all the settings are correct;

• attaching the assignment & supporting materials;

• assigning missing Assessors and Reviewers and allocating them to students. See ii below for recommendations on FLOWmulti (MCQ type exam) marking;

• adding additional time for any students with SORAs or ECs

Detailed guidance on these actions can be found under the Add section of the Manager Tasks table.

ii. Marking and review settings

The platform's marking and review settings determine how and when feedback and grades are finalised and released to students, amongst other things. Agreement on marking and feedback processes should happen at the Planning stage.

In particular, the sharing of written feedback made by markers must set up before marking begins - this information must be made available to the administrator when they are configuring the assessment and marking settings. Administrators are required to set the default sharing option for annotations and comments (i.e., making feedback available to co-assessors and/or students ) and cannot be retroactively amended once marking has been completed. Therefore, it is important to set feedback release correctly for your assessment needs and to communicate with markers and reviewers how this will work for them.

iii. Creating content on the platform (MCQs, Rubrics etc.)

If running an online exam (FLOWmulti) or using a rubric (marking scheme), these need to be created directly on the platform (see Create assignment and Create rubric). The person who creates this content on the platform is called an Author and this role can be assigned to either academics or administrators.

To complete this task, we recommend:

• Agreeing on a timeline for creating and approving assessment content and attaching to the flow;

• When creating your rubric, use the template rubrics provided to save time. In the Author role, on the Rubrics page, under Published Rubrics, search ‘Template.’ You can click on the menu option to copy a rubric to edit.

• Ensuring that there is contingency planning for ownership of author groups. You may want to collaborate on building questions or give access to externals or other colleagues to check and approve questions. In this case you will need to create author groups and add these colleagues to the group. If you create an author group, the owner is the only person who can add new members to that group;

• Keeping a master copy of question text (in a word document). You may have questions already on Moodle that can be imported into Assessment UCL (see Import questions).

• Considering how you will organize your online exam questions. On Assessment UCL, online assignments are structured in layers (from top to bottom); the Assignment ‘shell’ of settings > Items that act as pages or sections of your exam > and finally Questions. Items help structure your paper by theme and for the student they represent a page of the exam. If your exam is a MCQ exam, it is best to have one question per Item. If you have a series of questions under a theme then you could put them all into a single Item (group of questions). However, be aware that:

• you cannot randomise questions within Items

• student submissions will not save until they have completed all questions in that Item

• you cannot convert questions back into separate Items.

• ‘Promoting’ questions to the Content Bank, where you can create/share new items and add questions from there onto your Flow.

iv. Activating assessments

Once the flow has been set up correctly, with the relevant content created and attached, the flow must be activated (see Activate assessment). Activating an assessment means that students can view the assessment instance when they log into the platform but will not be able to access the assignment content until the participation start date/time. Upon activation, students will receive an email notification with the date/time of their assessment and a platform link to the assessment.

Departmental flows are activated by the person with the Manager role, e.g., Exam Liaison Officer or teaching administrator.

v. Messaging via the platform

As mentioned, students will be notified by email when the Manager has activated the assessment on WISEflow. Students will also receive a reminder message 24 hours before the start date/time of the assessment that they are due to take the assessment on the platform.

Students are not notified when marks and feedback are released on the platform – for this, and other notifications, you may wish to schedule a platform message in advance (see Send messages via WISEflow).

4. DURING THE ASSESSMENT

As with any departmental assessment, it is recommended that Administrators are on hand to monitor student progress and provide support where applicable (particularly for timed assessments).

i. Monitoring student progress

Administrators with the Manager role can track student engagement with an assessment in real time using the Participation monitor.

ii. Student support

When taking an assessment on WISEflow, students may contact you for assistance. Please direct them to the Support during your assessment page which outlines the various support that students can access at different stages of their assessment.

For example, if students are experiencing technical issues (i.e. logging in or submitting), and there is still time left before the deadline, they can contact the ISD helpdesk for support.

For other matters, there is an Assessment Query Form attached to every flow that students can access on the overview page of their assessment. Once submitted, these queries will be responded to by the central assessment team.

5. MARKING AND FEEDBACK

Marking and feedback is one of the main concerns for staff in the assessment process.  Completing the planning phase of marking and feedback (see PLANNING) will make life easier for markers and moderators and help the whole process run smoothly.  Planning how your department will mark is essential to consider as certain marking workflows work better than others, and others are not yet suitable for the platform (see examples under Marking an assessment: methods available).  Step-by-step guidance on marking can be found in the Mark section of the Marker (Assessor) guide.

i. Before marking period begins – we recommend that the Course or Module lead:

• Informs markers about assessment processes, timelines, training requirements and how to get support. Ideally hold a Teams or Zoom prep meeting.

• Agrees a standardised marking framework for all markers (where possible); this will make the management of marking much easier and guarantee that students are provided with the same depth of feedback.

• Understands marking options available (most of this would be discovered during the planning stage). Markers can mark using annotations, comments, a digital rubric and/or grades. They can choose to share any of their marking with students and/or keep some or all private for other markers - understanding this will avoid manual work re-sharing marking.

• Informs markers who the Administrator is (Manager role on Assessment UCL). If they cannot access assessments, it is most likely that they haven’t been allocated to submissions and the Manager can do this.

• If you have a lot of different markers, create a Teams site so that they can communicate, be supported as a group and all have access to key information and guidance in a timely way. Add your DAA to support.
  
  

ii. Double-blind marking

• If the assessment is double blind marked, you may not know who your co-assessor is. The system does not allow for first markers to alert second markers that they have completed marking. Therefore, you might want to agree on whether to mark in parallel or on a timeline for completing first marking.

• Specific recommendations for marking FLOWmulti exams (e.g. MCQ type exams)

Online exams (FLOWmulti) work best when set up for automatic grading. Departments can follow the Automatically marked exam (FLOWmulti Blueprint) to ensure a smoother delivery of this assessment type.  Manual marking should be avoided where possible, especially if you have several markers. Please note also that online exams cannot be automatically re-graded if an error or additional answers is found after the exam is run.

It is possible to view student performance statistics and to export student MCQ answers to make manual corrections in Excel. For more complex automatically marked questions (e.g. short answer, drag and drop) this approach is not recommended and will require thorough manual checking of answers if they need to be regraded.

For further guidance on recommendations, please refer to the FLOWmulti Blueprint.

6. POST-ASSESSMENT

Once marking is completed, there are several tasks the administrator in the Manager role must complete:

• Checking any late or word count penalties. Late submissions can be identified on WISEflow (see guidance under Identifying late submissions) but penalties must be recorded in Portico (guidance here).

• Notifying students that feedback has been released. They won’t get an automatic email but you can schedule a platform message in advance (see section 2v).

• Exporting grades to Portico. This can only be done once the marking to be complete and the marking period to be closed.

• Providing External Examiners with information and guidance. External examiners will have the Reviewer role and can be allocated to a sample or all papers.

• Archiving/un-archiving. Flows get archived and become read-only 30 days after marking closes. They can still be viewed by students, markers and external examiners in their respective roles, but they appear filed under ‘Archive’. The Manager can un-archive a flow to make changes e.g. add an external examiner. View guidance under View archived flows/ Un-archiving a flow.