M57 - STACK online assessment for mathematics and science

Keywords: quiz, question, mathematics, science, exam, multiple choice, e-exam, e-assessment, summative, formative, instant feedback

What is it?

STACK is a world-leading open-source (GPL) system for mathematical and scientific assessment. A STACK question is one of the question types that can be included in a Moodle quiz to formatively or summatively assess your students. 

Why use it?

• Separate validation and assessment allowing for complex formative feedback and line by line evaluation of student responses.
• Multi-part questions.
• Dynamic plots for question text and student feedback.
• Randomly generated questions.
• Support for scientific units and significant figures.
• Students type in mathematical expressions using a syntax called Maxima.

To see a variety of example STACK questions, request access to the STACK demonstration course by emailing digi-ed@ucl.ac.uk.

Who can use it?

• Tutors can create mathematical and scientific questions, and create response trees to provide detailed feedback to help students learn.
• Students can enter mathematical and scientific responses, see a preview of their answer before they submit, and receive feedback and grades for their working out.

Before I start...

Think about:

• Is the question text clear to students?
• Will students know how to input an answer? How familiar will my students be with Maxima?
  
  • Could a "syntax hint" or "validation" help the student?
• Do I want to apply a penalty for wrong answers?
• What are the likely responses to my question and how would I evaluate each in turn?
• Do I want to randomise my questions?

Meeting the baseline

The UCL E-Learning Baseline suggests the following for Orientation:

• 2.2 Explain participation requirements:
  • Identify which activities are compulsory and optional.
  • Provide an indicator of effort (such as timings or page counts) for all compulsory tasks.
  • Explain how students are expected to use UCL and external e-learning tools. This PowerPoint Induction template provides a starting point. Wholly online courses might provide this information as a screen-cast video, with a voice over.
  • Link to instructions for any e-learning tools that students are expected to use.
• 2.8 Outline the Intended Learning Outcomes for every activity and resource in the description and provide a clear overview of what the student is expected to do.

The Assessment category suggests that you should:

• 4.1 Provide an assessment outline with a clear description of the module assessment, including schedule, criteria and submission details. To avoid duplication this might include a link to a module webpage.

How do I set one up?

The STACK question type is quite complex, but the three main steps are:

1. Add question
2. Configure Potential Response tree
3. Test questions

The team behind STACK have created a series of videos that explain the basics of authoring STACK questions, the first video is provided below:

You can find further videos and guidance in the STACK Authoring quick guide and Digital education recommends completing this guide before running your first STACK question on UCL Moodle. 

Step 1: Add a STACK question (minimal requirements)

Within a question bank or quiz:

1. Click Add.. and New Question
2. Select STACK
3. Give the Question a meaningful and unambiguous Question name.
4. Define any Question variables using the notation: 'variable: value' e.g. exp: 3*(x-1)^(-4)'. It is essential to include a colon to assign the value to the variable. Avoid single-character names as they are meant for student input. 
5. In the Question text, provide the question as clearly as possible. You can call Question variables with the notation: {@variable@} e.g. {@exp@}.
6. Position the [[input: ans1]] tag appropriately. This is where the student puts their answer.
7. Position the [[validation: ans1]] tag appropriately. This is where any feedback related to the validity of ans1 will appear.
8. Under Input: ans1, enter a Model Answer.
9. Under Potential response tree: prt1, set
   1. Answer test to AlgEquiv
   2. Student Answer(SAns) to ans1
   3. Teacher Answer(TAns) to the Model answer (you may have this stored as a variable or you can enter the expression directly.
   4. Set Score or Penalty.
10. Click Save changes and continue editing

Step 2: Configure Potential Response tree

Consider the likely mistakes a student would make.

For every answer test, create a node:

1. Click Add another node.
2. Set Answer test.
3. Set SAns, TAns and Test option.
4. Set Score or Penalty.
5. Set Next value to achieve the desired traversal of tests. Note. You can click Save changes and continue editing to refresh the preview of the response tree, as pictured below:

Step 3: Deploy random variants

Questions in STACK can contain randomly generated elements. A student will be given a random variant of a question generated by a pseudo-random seed.

Why deploying variants

Questions in STACK can contain randomly generated elements. A student will be given a random variant of a question generated by a pseudo-random seed.

The tutor is strongly advised to pre-generate and "deploy" variants of a question. Not pre-generating question variants Forces Moodle to generate them on the fly - for quizzes with larger numbers of participants this can cause quizzes to crash/freeze.
When a student attempts the question, they will be given a random selection from the deployed variants.

Other reasons for deploying variants of a question:

• STACK runs all the question tests on each deployed variant to establish each variant of the question is working. This aids quality control. By using question tests, it is unlikely a student will be given a random variant which does not work correctly.
• The tutor can decide if each deployed variant appears to be of equal difficulty. The tutor can easily delete variants they do not like.

How to deploy question variants

The deployment interface can be found by editing a question and clicking on question tests and deployed variants.

1. The easiest way to do so is to preview the question:
2. Then click the Question tests & deployed variant link on the top right corner.
3. Click ‘deploy’ if not already deployed.
4. Next to Attempt to automatically deploy the following number of variants, enter the number of variants you would like and click Go. You can preview results and either exclude variants, or return to the quiz question settings to revise the randomisation you have used in the question.
5. Check variants as required.
6. This will show the list of currently deployed variants, and links to undeploy all or a specific variant.
7. Optionally, click ‘Run all tests on all deployed variants (slow):’ and check/undeploy any variants you don’t want to use.
   
   Limitations

Step 4: Test questions

You can test questions by deploying random variants, and adding test cases.

Add test cases:

1. Click Add a test case
2. Provide a test input, and the expected outcome.
3. Click Save changes
4. Clik Run all tests on all deployed variants to run the test on all the deployed variants of your question.

Caution

• Not pre-generating question variants Forces Moodle to generate them on the fly - for quizzes with larger numbers of participants this can cause quizzes to crash/freeze. Ensure you generate/deploy question variants before the quiz runs.
• STACK allows students to use standard mathematical functions including sin, cos, and int for integration. For integration questions, you need to add 'int' as a forbidden word or else students can enter 'int' and Moodle will perform the integration for the student.
• Students will need to be instructed on how to enter Mathematical equations correctly. Digital Education recommends having a formative or practice quiz to familiarise students with the process of completing and submitting a STACK question. Guidance on Maxima can be found on the STACK documentation page, and in the Maxima manual.
• When using randomization, ensure you run question tests to ensure the difficulty of the question is consistent. See the STACK documentation page on Randomisation for more details.
• Once a quiz is underway it is still possible in Moodle to edit a question, and to re-grade students' attempts. This is useful in rare cases where there is a mistake. However, do not change anything related to the random generation of questions! Results are unpredictable, and may well result in a situation when the modified question is different to that answered by students taking the test prior to modifications. 

Examples and case studies

Coming soon

Questions & Answers

How do I add plots and graphics to my question?

Please read the STACK guidance on plots and graphics.

Further information

This guide was adapted from the STACK Documentation website, which provides a detailed walk-through with examples and videos for the STACK question type. Accessed on the 14th February 2020.

Save

Save