M12i - CodeRunner question type

Keywords: quiz, question, exam, e-exam, e-assessment, summative, formative, instant feedback

1 What is it?
2 Why use it?
3 Who can use it?
4 Before I start...
5 Meeting the baseline
6 How do I set one up?
- 6.1 Step 1: Create a quiz
- 6.2 Step 2: Add a CodeRunner question
7 Caution
- 7.1 Further help
8 Examples and case studies
9 Questions & Answers
10 Further information

What is it?

CodeRunner is a Moodle question type that allows students to submit code and for teachers to run a program in order to grade a student's answer.

CodeRunner will be of most benefit to programming courses where students are asked to write program code to some specification and that code is then graded by running it in a series of tests.

CodeRunner questions are also relevant for areas of computer science and engineering to grade questions in which a program must be used to assess correctness.

UCL has support for the following common coding languages: Python, Java, C, SQL, PHP. To request a language please contact Digital Education (via the IT Services).

CodeRunner questions are an advanced question type. You should always run a practice assessment with students prior to any summative assessments to ensure your questions are working properly and that students are familiar with answering CodeRunner questions.

Why use it?

Allows for students to write code within a quiz or exam setting.
Allows for automatic marking of student responses based on predefined tests.

Who can use it?

Tutors
Course Administrators

Before I start...

Think about:

Do I want to apply a penalty for wrong answers?
Do I want to randomise my questions?
How can I prepare students to complete a coding quiz?

Meeting the baseline

The UCL Connected Learning Baseline suggests the following for Orientation:

2.2 Explain participation requirements:
- Identify which activities are compulsory and optional.
- Provide an indicator of effort (such as timings or page counts) for all compulsory tasks.
- Explain how students are expected to use UCL and external e-learning tools. Wholly online courses might provide this information as a screen-cast video, with a voice over.
- Link to instructions for any e-learning tools that students are expected to use.
2.8 Outline the Intended Learning Outcomes for every activity and resource in the description and provide a clear overview of what the student is expected to do.

The Assessment category suggests that you should:

4.1 Provide an assessment outline with a clear description of the module assessment, including schedule, criteria and submission details. To avoid duplication this might include a link to a module webpage.

How do I set one up?

CodeRunner is an advanced question type. You should consult with the following two resources before creating any questions.

Watch the Bath University Introducing CodeRunner video to learn more about how CodeRunner works. Please note that UCL does not currently support all the programming languages shown in the video.

Try out some test questions on the Coderunner test site.

When you are ready, you can then create a CodeRunner question on UCL Moodle.

Step 1: Create a quiz

You can find further videos and guidance in the Moodle guides and we recommend reading Guidance on Building a Quiz before creating your first quiz on UCL Moodle.

Step 2: Add a CodeRunner question

Within a question bank or quiz:

Click Create a New Question or Add a new question.
Select CodeRunner
CodeRunner is an advanced question type. You should consult the in-line documentation while authoring CodeRunner questions - just click the ? help icons.
At the very minimum you will need to complete:
- Question type: select the appropriate coding language
- Question text: provide students with the question prompt
- Answer: provide the correct answer.
- Test case: provide test to evaluate the student's response
- Expected output: the output expected by the students answer when the test case is run.
You should also be aware that CodeRunner questions allow students to check their responses at the cost of a penalty if their answer fails. As explained in the CodeRunner documentation:

Regardless of the behaviour chosen for a quiz, CodeRunner questions always run in an adaptive mode, in which students can click a Check button to see if their code passes the tests defined in the question. If not, students can resubmit, typically for a small penalty. In the typical 'all-or-nothing' mode, all test cases must pass if the submission is to be awarded any marks. The mark for a set of questions in a quiz is then determined primarily by which questions the student is able to solve successfully and then secondarily by how many submissions the student makes on each question. However, it is also possible to configure CodeRunner questions so that the mark is determined by how many of the tests the code successfully passed.

You can change this behaviour under the Marking setting when editing a CodeRunner question.

Click Save changes and continue editing to add more questions.

Caution

Students will need to be instructed on how to answer questions correctly. Digital Education recommends having a formative or practice quiz to familiarise students with the process of completing and submitting a quiz attempt.

Further help

Further guidance on the CodeRunner Question Type is available on the official documentation page.

If you find any inaccurate or missing information you can even update this yourself (it's a communal wiki).

If you have a specific question about the tool please contact Digital Education (via the IT Services) for advice.

Examples and case studies

Coming soon

Questions & Answers

-

Further information

N/a