Images

Please note, images in the guidance below can be expanded by clicking on them.

The manager role is designed for staff who are responsible for the administrative tasks associated with the assessment process. The manager creates the assessment or flow, and is also responsible for enrolling students, markers, moderators, external examiners and invigilators onto the flow.

The front page of the manager tab is the flow overview which lists all the assessment instances or flows you are managing, whether you have created them yourself or been assigned as co-manager. On the flow overview, you have a range of tool to help you sort and filter your list of flows to get the view you prefer to work from.

This takes you to the FLOWseries overview.
This takes you to the FLOWcombine overview.
The button Archived flows to the right contains previous flows which are no longer active, but which are stored in case you need to extract information from them.
Basic search lets you filter the list of flows based on title, subtitle, code or flow id.
Advanced search opens a window where you have more advanced features available to make your search even more precise.
Search for flows a specific user is associated with. The result will also show the person role on each flow.
Filter the list of flow based on a specific flow type chosen in the drop-down menu.
Show flows based on your relation to them. Choose between showing flows you have created or flows you have been added as a manager to.
Will open additional filters for the phase you are currently viewing.

10. Choose the phase you want the list of flows to show. The phases are representative of the state the flow is currently in.
11. This button opens the settings for the list of flows. It allows you to toggle which columns you want in the list and it will save your preferences, so you do not have to set them each time you log in.
12. Click Open to go to the flow or click the small icon to the left to open the flow menu that allows you to go straight to a specific subpage of the flow.

The required assessment type determines which flow type needs to be used. Examples of common assessments and the recommended flow types are listed below.

Assessment type	Recommended Flow type	Description	What the student submits	What the marker can record

Assessment type	Recommended Flow type	Description	What the student submits	What the marker can record
Open book exams, Essay submissions, Dissertations, reports	FLOWassign	Mainly used for course work or open-book examinations where the submission handed in is a PDF file. Group submissions are possible. Student groups can be random, populated from a spreadsheet, or selected by the student. Assessment feedback and grade goes to all students in a group, with the option to overwrite grades for individual students.	PDF file (up to 200mb)	Mark Summary Comment In-line annotations (e.g. highlighting) Rubric marking
Written unseen exams, Multiple choice question (MCQ) exams	FLOWmulti	Used for exams with multiple question types, including multiple-choice, math, interactive questions etc. The FLOWmulti assignment is authored in AssessmentUCL where 50 types of different question type can be used in various combinations. The FLOWmulti assignment can contain both auto scored and manually scored questions. The flow can be with or without the use of the lockdown browser depending on the type of exam.	Online exam with multiple question types including MCQ and Essay responses	For each section/question a marker can be assigned to provide: Mark. Summary Comment. Annotation comment.
Digital assets (video, audio, website)	FLOWhandin	Allows for students to submit files in various digital formats. Students can hand in formats such as PDF, Word, Excel, audio and video files etc. When the participation period ends, the files are made available to the assessors to be downloaded and reviewed.	One or more files up to 5GB	Mark Summary Comment Rubric marking
OSCE or Viva timetabling	FLOWoral	Enables administrative support of oral exams. Exams can be scheduled over several days with the associated students and markers allocated to the exam schedule. Markers can see a list of students and the exam schedule. The marker can take notes digitally on the AssessmentUCL platform as the student presents either in person or via a web conference platform such as Teams or Zoom.	Nothing submitted in AssessmentUCL, students are timetabled	Mark Summary Comment Rubric marking
Invigilated written unseen or Multiple choice question (MCQ) exams	FLOWlock	The student’s computer is secured, allowing only the use of the FLOWlock Text Editor, preventing the use of aids during on-site exams. The students write their assignment in the editor and upload it within the time frame (participation period). Students can also be allowed to capture hand written responses with their webcam, and specific websites or resources if required. Please note that the locked down browser is typically for invigilated exams. Getting students to use a locked down browser without invigilation does not guarantee they will be unable to access additional resources, e.g. they can simply use another electronic device.	Online exam in a locked down browser	Mark Summary Comment In-line annotations (e.g. highlighting) Rubric marking
Combinations of the above e.g. a report submission using a FLOWassign followed by a media submission using FLOWhandin	FLOWcombine	Allows for the creation of a main assessment and the subsequent addition of additional assessments. FLOWcombine makes it easy for students, markers and moderators to be added to multiple assessments.	Combination of file types	Depends on flows that are used
Similar to FLOWcombine, with the addition of a final assessment.	FLOWseries	FLOWseries supports course assessment that consist of several assessments that culminate in a final exam (a FLOWmulti). A series of assessments can also be combined to generate a final "scoreboard" where component marks can be totalled to reach a final mark. This is similar to the Gradebook in Moodle.	Combination of file types, and online exam	Depends on flows that are used
Attendance registration	FLOWattend	FLOWattend allows recording the attendance of students.	Nothing submitted in AssessmentUCL	Mark Summary Comment Rubric marking

Create assessment instance (flow)

Assessment (flows) will be automatically created via Portico integration. When a manager logs into AssessmentUCL they will be able to view and search for the module related to the assessment instance.

Basis settings

Pre-populated fields:

Assessment title: Flow title
Assessment sub-title: Flow sub-title
Students attached to an assessment instance: participants on the flow
Flow type; e.g Assign or Multi
Participation Start and End date/time
Assessment Start and End date/time
Anonymous participants - set to Candidate number
Marking scheme: Assessment type

Manual updates

A manager needs to manually upload the following sections/fields of a flow (assessment instance):

Attach assignment (exam paper) - To be handed out tab
Cover sheet: add fields to the Cover sheet
Add associates to a flow: e.g. author or manager
Flow settings; e.g. marking method
Add markers and moderators to a flow
Attach Rubric to a flow
Anonymous internal assessors

The AssessmentUCL platform is integrated with Portico. When you are creating a new flow, you will need to select a relevant module from Portico/SITS to pull the data through from Portico. This will ensure that the following data is pre-populated using Portico data:

Module code
Assessment pattern
Students attached to each assessment component

How to create a flow

Click on the green button Create in the top right-hand corner and select New flow.

From Choose source list: select SITS Assessments

Enter Module code and Academic year - > click Fetch list

From the list select the relevant assessment instance → click + Select button.

Example:

AMER0038 = module code
A4U = Level e.g. undergraduate
Subtitle = method of assessment e.g. coursework
2020 = academic year

Click Show source data button to check all the Portico data is correct; e.g. participants (students).

Upon opening an assessment instance (flow), a Manager can edit the following:

Basic settings

1 Basic information:

The first quadrant, called Basic information, contains various fields: you can view the flow type, set the subtitle,

2 Deadlines:

Under Deadlines, the participation and assessment periods are set.

In the participation period, the participants can access the exam assignment and submit their answers.

In the assessment period, the assessors can assess the submitted papers and submit grades. If the manager has checked the option of showing the marks to the participant in AssessmentUCL, the marks will be visible when the assessment period ends.

3 Details:

Details tab contains different fields depending on the type of flow that you have created.

Here is an example with FLOWassign:

Anonymous participants: this field allows you to select whether you want the students to remain anonymous to markers. There are several identification methods to use; e.g. Candidate number or UCL user id. Select Candidate number to ensure students remain anonymous to markers during the marking process.

Anonymous internal assessors: determine whether the markers should be visible to students.

Participants: are allowed to upload appendix material: check this box if you want to allow students to upload any additional material to their main assessment paper.

Individual assignments: this feature allows you to distribute individual assignments to each student while the general assignment (if uploaded) will still be visible and accessible to all students.

ECTS: leave this field blank, it is not in use by UCL.

Assessment type: select the marking scale to be used by markers when marking an assessment. This field should be integrated from Portico, and only overwrite it if your assessment has a different marking scheme.

Group type: If you want the participants to hand in assignments in groups, you can decide if they should hand in one assignment each or one assignment collectively.

4 Assessment and review:

Under Assessment and review section, you get an overview of the assessors, rubrics and reviewers on the flow and have shortcuts to managing these.

Internal assessors are the people from within the institution responsible for assessing and grading the participants’ submissions; e.g. markers.
External assessors are the people from outside the institution who assess and grade the participants’ submissions; e.g. external examiners.
Reviewers are people with access to the assessments and are useful if you want someone to monitor the assessment process and the quality of the exam; e.g. Module Leader.
Rubrics can be attached so that it is available to all markers.
Additional materials can be provided to markers and reviewers e.g. marking criteria, model answers etc.

Participation settings (FLOWlock and FLOWmulti)

Flows that allow the use of the FLOWlock browser are equipped with another set of settings called ‘Participation settings’. These settings affect the FLOWlock browser in an exam for individual participants or across all participants on the flow.

The To be handed out tab contains options about the material that is to be distributed to students and assessors and reviewers when the flow is activated or when the participation period begins.

Assignments

Here you can set up the assignments on the flow and manage them. Alternatively, you can assign an author to the flow who can create and attach the assignment. The assignment document is released when the flow opens for participation. The appendix material are documents that support the assignment, e.g. tables, figures, articles etc. These are released along with the assignment.
When you click on the green + you will be taken directly to a window where you can create a new assignment and upload files to it.

Name the assignment
Choose to allocate this assignment to all participants by default or use manual distribution.
Upload the assignment using either drag and drop or the browse button.
Upload one or more appendix material using either drag and drop or the browse button.

Additional material for students

Additional material can consist of URL links and/or additional files. These will be available when the flow is activated. Items added here will only be shown to the students.

General material

Here you can provide participants, assessors and reviewers with material that is relevant for all of them. The material will be available when the flow is activated.

FLOWassign

The following options are available for FLOWassign assessments:

Procedure for submitting final assessments: select either Approval by Reviewer (final marks to be approved by Module Lead) or Agreement between co-assessors (e.g. double marking method)

Submission of assessment: ensure this is enabled.

Show final assessment: ensure this is disabled. We are releasing final marks in Portico for students to access. If an assessor selects the option to share their feedback with a participant, a student will be able to view the feedback in their AssessmentUCL pages.

Explanation support: ensure this is disabled. Student will not be able to contact a marker or manager to request and explanation for their submission.

Enable similarity report: ensure this is enabled. If you disable this option, Turnitin similarity report will not be available to markers assigned to a flow.

FLOWmulti

Section based assessment: enable this setting to allow section based marking. This feature enables different sections of an assessment paper to be allocated to different markers for individual marking.

This setting is not available in FLOWassign.

On the Basic settings tab of your flow go to the Assessment and review section.
Click Add/remove assessors button

This will bring you to the following screen:

Add assessors to the flow by clicking the Add assessor button. This will open a new window, where you can search for one or more assessor and add them to the flow. After selecting the assessors, you have the option to add them as internal or external assessors.

Once they have been added to the flow you get the following information.

In addition to their name, email, assessor type and allocation stats, you have the following actions available.

Remove the assessor for all participants.
Allocate the assessor to all participants.
Edit the assessor type.
Remove the assessor from the flow.

Allocate assessors to participants

Click the Allocate the assessor for all participants button (labelled 2 in the image above) to allocate a marker to all submissions.

AssessmentUCL offers an assessment method for FLOWmulti, called section-based assessment. This feature allows the manager to assign assessors (markers) to score individual sections of an exam paper.

Under Assessment and Review, click Manage settings.

Within the Assessment and review settings, you can enable section-based assessment by clicking on the pencil and filling out the checkbox for this setting.

The FLOWmulti is now set to Section-based assessment mode, and you can now edit the matrix by going back to the assessment and review page, where a new option will have appeared. Manage matrix will let you edit the matrix and distribute your assessors to the various sections on your assignment.

Note: The 'Manage matrix' button will only be available if there is an assignment on the flow, otherwise it will not work as intended.

If you have already allocated assessors to the flow then click on the Manage matrix button shown above. If not, assessors must be added first by clicking on the Manage assessors button also shown above. Then, if having just added assessors, you are still on the Assessors page, click on this Section based assessment setup button:

Both this button and the Manage matrix button opens the Manage matrix page. In the Scoring matrix section of this page you can allocate sections of an exam paper to an individual marker, by clicking on the relevant box.

Note: An Author must set up assessment as FLOWmulti assignment with multiple sections within the exam.

The scoring matrix allows the manager to:

Deactivate sections so that they are not assessed; e.g. Section 1. Note: The section will still be present in the exam.
Set the question to be automatically validated by the system; by clicking on auto-scoring. Note: the question must be assigned a score and marked as available for auto-scoring.
Set maximum and minimum points that can be scored for sections validated by assessors.
Assign sections to assessors or assessor groups.

The manager can add and remove a rubric from the flow. Rubrics can be added to all flow types except FLOWmulti.

Go to theAssessment and review to add and manage rubrics on the flow:

Add the rubric to the flow

Rubrics are published by authors and can be attached to the flow by the manager or author. The manager has two options:

Choose the rubric from a list of shared rubrics on the licence.
Attach the rubric by direct reference.

The author can add the rubric to the flow if the author is associated with the flow.

The manager can regulate settings regarding the rubric on the flow:

Assessment settings

When the mark conversion is enforced, the participant's final mark is dictated by the rubric score and the preconfigured mark converter. The assessor will not be able to manually set a differing final mark.
When enabled the rubric is set to be shared by each assessor. The individual assessor can still decide to not share the rubric.

Participation settings

This allow the participants to preview the empty rubric from the time the flow is activated.

Mark converter

A mark converter on the flow can convert the points achieved on a rubric to a final assessment.

The author can add a mark converter to the rubric in the author tool when building the rubric. If the flow has the same assessment type as the type the author has chosen, the manager can choose to use it. If the mark converter is the same on the flow, it will be visible with a mark converter icon:

The manager can also change the mark converter from the author or add a new converter to the flow.

Change the mark converter by dragging each interval border or by entering the boundary values of the intervals directly:

If you change your mind about the changes you have made to the mark converter, you can go back to the original from the rubric (made by the author):

Find a guide that describes what a mark converter is by hovering over the ?

When you have created a flow, you can add students/participants. When you click Add participants, you get a popup with two options:

Add participants one at a time by searching for a name, email, or user ID.

The participants are now visible in the participant overview at the bottom of the screen. Here you can see further information about the participants and edit their individual participation periods.

Add participants in bulk by searching for email, CPR, Institution-ID, or IDs.

Coming soon

Under Associates tab, you can add users with other roles to a flow. This will enable those users to perform additional functions on an associated flow.

Author: you can allocate an author to directly assign the finalised assignment to the flow; you will not need to complete this step.

Manager: you can allocate additional members of your team as managers to a flow.

Invigilator: we are not using Invigilator role for the 2020-21 academic year.

Timed exams duration is integrated from Portico. Important: do not set up these manually or override the existing values if you have created a flow via SITS integration.

The timed exam is indicated by the blue stamp Individually timed flow duration on a flow home page.

Flow Assign

On Flow Assign you can view the exam duration on the following screen:

Open book 24 hr exam will not have a duration allocated at Individually timed flow duration field:

As the manager of a flow, you have access to the message centre. The message centre is an overview of the different e-mail messages that have already been sent out and e-mail messages that are scheduled for a later point in time. Furthermore, the message centre gives the manager the opportunity to see the content of these e-mail messages.

As the manager, you can also send out a custom e-mail message to everyone associated with the flow or specifically for each user group. The licence administrator also has the opportunity to change the contents of the standard notifications that are sent out.

Where to find the message centre

The message centre is found in the upper right corner of your flow, where you your flow.

How to understand the message centre

The message centre is a timeline with its beginning at the top and with the dates for flow activation, participant start, assessment start, assessment end and so forth. Messages are symbolised by boxes.

The title (Participant: When flow is activated) shows what user group the message is meant for, and at what event the message was sent out. Below, is a short description and the date the message was sent out. Click the eye icon in the top right corner of the message to show a preview of the exact message sent out.

Messages are situated on the left or the right side of a dividing line. Messages on the left have been sent; messages on the right are messages that have been scheduled, but where the flow is not yet in the state where the message is sent.

The blue dot with Now indicate what state the flow is in right now. The flow shown above has just been activated.

Messages that have already been sent out shows a small warning if there were no recipients. Users added to the flow after a message has been sent out will not receive the message. To ensure that users receive all messages related to the flow, verify that all participants, assessors etc. are correctly added and assigned before activating the flow.

How to deactivate a scheduled message

In some cases, you may want to prevent messages from being sent. This can be done for all scheduled messages by clicking the small menu button.

This opens a small window where you can choose if the message should be sent or not and then save the setting.

All scheduled messages that have been disabled will be shown in the message centre with strikethrough on the text

How to send a custom e-mail message

Click Create email message button.

When you enter the compose message screen, there are 3 steps you need to go through.

1. Compose a message

You can write ordinary text and insert dynamic fields in your message. Dynamic fields are automatically converted to the names, titles, dates and times and so on, of the flow in question.

2. Choose recipients

After you have written your message, you have to choose the user groups the message should be sent to. You can mark one or more of the groups by clicking on the circles. Chosen roles will have a line around the circle to indicate who the message is sent to. It is also possible to mark all groups at once by clicking the All relations circle.

3. Send or schedule

The last section makes you choose between sending the message immediately or schedules a date. After choosing your option you can click on the button Send message to finish the custom e-mail message.

When you have created a new FLOW, you must ACTIVATE each individual flow by clicking on Activate flow button.

Important: students and markers will not be able to view the assessments until a flow has been activated.

Students will receive automated email notification of the upcoming assessment/s; upon the flow activation.

Manage assessment instance (flow)

The FLOWmonitor is a tool where you can gain detailed information on the activities of the students enrolled attached to a flow and on the progress of their assessment.

A manager can access the FLOWmonitor from the Basics settings tab - > Participants section.

Overview: List

When you open the FLOWmonitor, there are several features available:

Identification: Three columns showing participant index, first name and last name.
Deadline and status: Two columns specifying the participant's deadline (updated in case of extended time), and the last time the participant sent information to the server.
Progress: Whether the flow page or FLOWlock has been opened by the participant (not the current status)

The participant has not opened the flow page.
The participant has opened the flow page.
The participant has opened FLOWlock.

The current progress of the participant. This progress bar differs according to the flow type.

The number of characters that have been saved to the server (FLOWlock).
Paper not uploaded (top) or paper uploaded (bottom) (FLOWassign and FLOWhandin)
Participant progress with the exam questions (FLOWmulti).

Participant hand-in status.

The participant has handed in.
The participant has not handed in.

Actions: For each participant, the manager can mark the participant as present or absent, add comments, extend the individual submission time and open the log.

Click to mark the participant as present or to mark the participant as absent. A participants attendance-status is shown by the colour of the buttons.
Extend the time on a single participant by clicking .
Access the log to see the actions of the participants. The detailed log shows a table of three columns:
- 1. Time stamp: The precise time of the activity
- 2. IP-address: The public IP-address of the network from which the AssessmentUCL server has received the information/activity
- 3. Message: Short description of the activity received.
Add a comment. This is also visible for the invigilator on the flow.

FLOWmonitor overview

You can change your view from list to overview in the FLOWmonitor. FLOWmonitor overview gives managers a more compact overview of participant statuses on the flow.

FLOWmonitor features

These are the features in the FLOWmonitor:

Filter the table by opened, paper saved, handed in, present, absent, active and inactive.
Choose what to show in the menu of the table: participant index, first name(s), last name, room, deadline for the participant, latest status update, progress and action.
Set the inactivity time (this will be remembered on the flow)
See all participants on the flow in one list. You do no longer need to go to the next page.
Go to the direct messaging page, where messages can be sent directly to all active participants on a flow.

To extend a student's individual participation period; go to Flow Monitor - > available on the Home page of a flow.

Find a student. You can search for a student using a search function or Filter it by process stage.

You can filter students by a stage in the process:

Click on the clock icon - > this will allow you to extend student's individual participation period.

Enter the new participation end date or time.

The extension will be highlighted in blue colour.

In the event that the wrong final mark is submitted or if there is a need to correct the given assessment, the manager has the ability to reset the assessment or un-submit individual marks.

A manager can un-submit a submitted mark for an individual candidate by opening the flow as a manager and clicking on the icon for data entry ("diploma") in the list of participants. The mark is then un-submitted and markers or a moderator with assessment approval privileges can edit the participant's final mark again.

If the assessment is set up with Approval by Reviewer, any marking submitted by a marker will not be deleted, only the final mark which the Reviewer submitted will be un-submitted.

Reset all assessments

If the manager resets assessments, then all marking is deleted i.e. all marks entered by each Assessors (markers) and the final marks approved by reviewer. Only feedback comments and annotations are retained. As a precaution, the Module Lead should download all marks before requesting the manager resets marks.

If all assessments have been submitted by mistake or there is a need to reset all them, it can be done by clicking the menu button with the 3 lines at the top left side of the flow. And then choose Reset assessments.

Note: This will also reset any personal assessments that have made, but may not have submitted yet!

To export marks from AssessmentUCL, go to Participants sections and click on Assessment tab. Once all the marks have been submitted by Assessors and Reviewer, you can export the marks into csv document. Marks are recorded under Assessment column.

Go to Tools button