Open

AssessmentUCL (WISEflow) is a dedicated end-to-end assessment platform with Portico integration, aimed at enhancing the design and delivery of assessment at UCL.

Link to AUCL (WISEflow):  WISEflow - More than paperless

This guidance outlined on this page covers the functions of the Reviewer role in AssessmentUCL (WISEflow), relevant to staff who moderate or review assessments (including UCL External Examiners).

Overview

What can a moderator (reviewer) do?

All moderators (Reviewers) can:

• View candidate's enrolled on an assessment by candidate number.

• See if a candidate has handed in or not.

• See Turnitin similarity reports.

• View all marks submitted by markers.

• View all marker's comments and annotations in the shared summary.

• Print out annotated submissions one at a time.

• Add comments and annotations.

External Examiners, and most internal moderators are likely to only need the standard reviewer role.

Some reviewers can also submit final marks. This ability is something that a lead internal moderator, or Module Lead would perform. For this option to be available, the manager must have created the flow with the 'Reviewer approval' method of determining final grades

[Back to Top]

Navigating the AssessmentUCL (WISEflow) platform

Open an assessment on AssessmentUCL (WISEflow)

1. Login to AssessmentUCL (WISEflow) (WISEflow) at https://uk.wiseflow.net/ucl

2. Click on UCL.

Open

3. You will be taken to the UCL sign in page, please enter your UCL username (remember to add '@ucl.ac.uk) and your password.

4. You will arrive on the AssessmentUCL (WISEflow) dashboard. Click the reviewing tab (Note. you may have access to more than one roles) 

5. You should arrive on the reviewer overview

In the overview, you can see a list of all the activated assessments or flows to which you have been added as a reviewer, and you can quickly get an overview of the assessment titles, subtitles and assessment deadlines. The list can be sorted for each column in alphabetical order or by newest/oldest. You can also find specific flows by using the search bar in one of the columns.

From the overview, you can also access the archive containing all of the previous assessments you have been assigned to as a reviewer and access active flows by clicking the open button.

Navigate the reviewer page

When you open an assessment in AssessmentUCL (WISEflow) as a reviewer you will be taken to the reviewer page.

If you have are a reviewer without special privileges you will see the following:

The main options on this page are:

1. See the assignment and additional material for assessors.

2. See the start and end dates of the participation and marking periods.

3. Click to see a list of flow managers.

4. This shows which procedure for the submission of assessments is used on the flow.

5. Filters the list by assessors.

6. This shows the percentage value of the similarity analysis; you can click on the figure to open the detailed similarity report.

7. Here you can either open the paper in the marking tool or go directly to the summary for the participant.

8. This column shows the assessment given by the assessors assigned to the participant.

If you are a reviewer with final grade privileges

When 'Reviewer approval' has been chosen as the marking process by the manager, one of the reviewers can be granted privileges to submit the final grades. The assessors will still assess and submit their assessments and, if they agree, this can result in a suggested assessment. However, it will still be the reviewer who has the final say.

1. See the assignment and additional material for assessors.

2. See the start and end dates of the marking period.

3. Click to see a list of flow managers.

4. This shows the procedure for the submission of grades on the flow.

5. Filters the list by assessors.

6. In the menu, you can download all papers as a ZIP file, export all the information to an Excel spreadsheet that can be used for offline marking and import the updated spreadsheet to AssessmentUCL (WISEflow).

7. By clicking Set all, you can set all suggested grades as final and Submit all will submit all the final grades to the manager.

8. This shows the percentage value of the similarity analysis; you can click on the figure to open the detailed similarity report.

9. Here you can either open the paper in the marking tool or go directly to the summary for the participant.

10. This column shows the grade registered by the assessors assigned to the participant.

11. If the assessors agree on the grade, it will be suggested to you and shown in this column.

12. By clicking on the box, a small window will appear that allows you to select an grade and either set it for now or submit it at once.

13. This will show all the submitted grades.

14. By clicking on a grade in the entered column, you can see who it was entered by and the padlock button cancels the entered grade.

[Back to Top]

Navigate a FLOWmulti as a reviewer

If you are added as a reviewer on a FLOWmulti and have been allowed to see or submit the grade, it will look a bit different than the other flow types.

1. See the assignment and additional material for assessors.

2. See the start and end dates of the marking period.

3. Click to see a list of flow managers.

4. This shows the procedure for the submission of grades on the flow.

5. Filters the list by assessors.

6. Here you can either open the paper in the assessor tool or go directly to the summary for the participant.

7. This column shows the sum of marks given by either the automated scoring or the assessor.

8. By clicking the points in the column, you can open a small overview of the section, and here you can cancel the assessors’ submitted points by clicking the unlock icon.

9. This section will only be available if you are allowed to submit the final assessments.

   1. Opens the grade scaler, which will be used to convert marks to a grade.

   2. Submits all the available marks.

   3. This column shows all the grades that were converted from marks in the previous column.

   4. Here you can see all the marks that are submitted.

[Back to Top]

Reviewing an assessment

View marking, comments and annotations

Reviewers can see all the annotations and comments left by markers, included those that are private, shared with co-assessors or shared with the student.

From the reviewer page, click the shared summary icon.

This will open up the exam submission and show all annotations down the page.

Summary comments left by markers appear above the student's submission.

The marker's name is identified, in the example above the markers are 'Demo Marker 1' and 'Demo Marker 2'. The visibility of the comment is also shown e.g. 'Co-assessors' indicates the comment is visible to all co-assessors, 'Private' indicates only the marker and reviewers can see the comment, whereas 'Participant' would indicate the comment will be visible to students when the marking is released. Marker's control these visibility options, see the Marker (Assessor) guide.

In-line annotations also show where they have been added to a submission. Comments appear to the right of the paper and can be clicked to focus on them and view all comment text.

[Back to Top]

View mark history (grade audit trail)

Actions made by assessors, reviewers and managers are recorded in the Grade Audit Trail. Reviewers access the Grade Audit Trail via the flow page:

Only submitted grades are visible in the Grade Audit Trail. Entered grades will not be displayed.

If the grades from multiple assessors have been submitted but there is a conflict between the co-assessors, the conflicting grades will be displayed in the Grade Audit Trail.

Any comments submitted to accompany the submitted grades will be displayed. So too will any Alternative Grades (please see Grade Audit Trail in the article Marking Overview (Submission of Marks and Grades).

The Grade Audit Trail will also show if an assessor has withdrawn their submitted grade (please see Withdrawing a Grade in the article Marking Overview (Submission of Marks and Grades) or if a reviewer or manager has unlocked a submitted grade.

[Back to Top]

Download annotated submissions

The following download options are available for each student submission. It may be necessary to combine the options below depending on what needs to be downloaded.

[Back to Top]

Export original submission

This option is within the Offline marking button - see 'Bulk Downloading scripts' section here for more info.

Print the Summary

From the reviewer page, click the shared summary icon.

This will open up the exam submission and show all annotations down the page.

If you print from your browser the Summary page, you will get the summary comments and the student submission, but annotations will be shortened making any text comments associated with annotations un-readable.

Print the Annotation view

If you click Annotation view, you can print out annotations but each annotation appears on a separate page and pages with multiple annotations will be duplicated. Pages without annotations will not be printed.

1. Click Annotation View at the top of the page.

2. Select thumbnail size.

3. Click Print.

4. In your print settings select Save as PDF.

Please note at present it is only possible to download annotated submissions one at a time.

Marker (assessor) exports their own annotations

This can only be done by the marker. See 'Export submisison with feedback' in the Marker (assessor) guide.

[Back to Top]

Add a comment (and upload feedback file)

To add a comment to a submission.

1. Open the submission in the marking tool (green icon consisting of a text bubble):

This will open the student's submission for you to add an annotation or add a comment as explained below.

2. Click the Candidate tab (labelled 1 in image below).

3. Click Add comment (labelled 2 in image below).

4. You will then be presented with an option to i) add a comment, or ii) upload a file (you can select the Upload button and upload a file without needing to add a comment).

5. Once added, ensure that the comment is shared with the relevant people (the sharing option is highlighted in orange). You can share the comment/feedback file with co-assessors and/or the student.

[Back to Top]

Offline reviewing features

Submissions made to FLOWassign or FLOWlock can be downloaded in bulk in a single zip file and can include a grading spreadsheet or grade sheet. To build your offline marking package, select Offline marking from the flow overview page.

To access the grade sheet, click here.

For more information read through the WISEflow article, Offline Functionality.

Export grade sheet 

Note: the below steps only work if you have access to the Reviewer role with 'decide final grade' privileges. Please contact your administrator (the Manager of the assessment) if you do not have the correct access.

1. You can export a grade sheet to complete the final grades for students in an Excel spreadsheet.

2. You can export a completed grade sheet after marks have been submitted by assessors to quality check the marking (this offers the same detail but a slightly different view to what you would see as a Reviewer in the browser).

Step 1: After selecting Offline marking, untick all boxes but leave the 'Include grade sheet' ticked:

Step 2: Select Export tab and select Download grade sheet:

Step 3: You will be asked to save the Excel spreadsheet. Once saved, you can open the spreadsheet to see a list of marks submitted by markers. If you are exporting the grade sheet before markers have submitted marks, the grade sheet will be blank (only the columns and candidate details will be shown).

[Back to Top]

Download all student submissions

1. From the reviewer page, click the menu button.

2. Click Download papers as a ZIP file. Please note the options to Export to spreadsheet and Import from spreadsheet only appear to reviewers with the privilege to decide final grade.

3. Choose whether you want the files downloaded in separate folders or one folder, and if you want the cover sheet inserted as the first page or included in a separate folder. 

4. Click Request Zip. AssessmentUCL (WISEflow) will now start generating a ZIP file for you to download. The time needed to generate the file depends on the workload on the server and the size of the file you have requested. You can close the window and return later by clicking Download papers as a ZIP file again.

5. When the ZIP file has been generated, the window will change and allow you to download the ZIP file. The ZIP file will be available for download for 24 hours. If the deadline expires, the file must be generated again.

[Back to Top]

Turnitin similarity checking

Turnitin reports are ONLY available for FLOWassign assessments. 

Turnitin reports can be accessed from the reviewer page or the marking tool which a reviewer can also access. Simply click on the percentage score to open the report.

Reviewer page example:

For guidance on interpreting Turnitin's report, please see Turnitin's help guides.

[Back to Top]

Reconciling marks

Methods of reconciling marks

There are two approaches to reconciling marks on AssessmentUCL (WISEflow).

• Marker agreement: markers must agree and submit the same grade*

• Reviewer approval:  markers can submit different grades which a nominated Reviewer then reconciles. The nominated reviewer could be one of the markers or the module lead

The Manager can set which approach is used for any given assessment or flow, but they should do so before marking has started.

*IMPORTANT: For Marker agreement, see the Marker (assessor) guide.

Video Summary: Reviewer approval

Submit final grades

Please note you will only be able to submit final grades if the manager/administrator has assign you with the 'Decide final grade' privilege. Contact your administrator if this is not set up correctly.

IMPORTANT: You must quality check the final agreed grades that are entered BEFORE submitting these grades. Once you submit the final grades, it is not possible for you to unlock/amend these grades.

Steps:

1. If both markers have submitted the same marks, skip to step 4. If there is a difference in marks, the Reviewer will need to decide the final grade: Click in the Final column for the student

2. Enter the mark in the pop-up menu

3. Click Set for now:

4. To submit all set grades in one go, click the green Submit all button (you may need to select the blue Set all button before submitting):

5. Once you have submitted the grades, they will appear in the Final grade column. This is the end of the marking process. Students will receive their grade at the end of the marking period. 

[Back to Top]

Unlock a grade for re-marking

Only reviewers with the privilege to decide final grade can unlock student grades and rubrics

To do so:

1. Click on the grade to open up a display showing who the grade was entered by.

2. Click the padlock icon to unlock the grade (Note. this does not delete any marking it simply allows the marker to change their grade and re-submit it). 

[Back to Top]

Deadline for reconciling marks

Reconciled marks must be submitted before the marking period ends. You can see the close date on the reviewer page.

The marking period can be extended by the manger of a flow.

[Back to Top]

Additional guides & FAQs

Export or Import grade

Reviewers can export a spreadsheet of data which contains columns for the final grade.

Steps: To export a grade spreadsheet

1. Select the Reviewing tab.

2. Select Offline marking:

3. Make sure 'include grade sheet' option is selected and select Export then Generate package.

4. Open the grade sheet and enter or transfer your final marks into the Final column. The exported spreadsheet can then be re-uploaded to WISEflow (Step 6).

5. Save file in the original format (i.e. Excel Workbook format (.xlsx); there is no need to convert the file to csv format as with other exporting/importing software you may have used).

6. To import

   1. Return to the Reviewing page.

   2. Select Offline marking then Import grade sheet.

   3. Select or drag/drop the spreadsheet exported from WISEflow for this assessment.

   4. Confirm the grades have uploaded correctly. There should be a message at the top of the table, and grade should appear next to the relevant students.

   5. Edit grade if required and don't forget to Submit grades when marking is complete.

An example of an update message where grades have been imported is shown below.

Make sure you click the green Submit button to confirm the final grades (once the marking deadline ends, students will see this final grade when they log into the platform).

[Back to Top]

View FLOWmulti statistics

Detailed statistics are available for a FLOWmulti assessment to the Reviewer.

The Reviewer can view statistics by selecting the statistics button from the flow landing page.

To learn more about the statistics available, please see WISEflow's FLOWmulti statistics guide.

The landing page shows a graphical overview of the generated statistics, split up into several sections which are explained further below.

[Back to Top]

Why can't I moderate?

You may not have been assigned to the assessment on AssessmentUCL (WISEflow), or you may not be assigned to the correct submissions within an assessment. 

If you need to moderate a Centrally Managed Exam, please contact the Exams team.

[Back to Top]

Need to speak to someone or having issues?

If you can't find the answer to your question in the guidance above, please raise a query via the staff support page.

[Back to Top]

Images

Please note, images in the guidance below can be expanded by clicking on them.