What are Multiple Choice Questions?
Multiple choice items consist of a stem and a set of options. The stem is the beginning part of the item that presents the item as a problem to be solved, a question asked of the respondent, or an incomplete statement to be completed, as well as any other relevant information. The options are the possible answers that the examiner can choose from, with the correct answer called the key and the incorrect answers called distractors (http://en.wikipedia.org/wiki/Multiple_choice).
Tips for setting up and fine-tuning a Moodle quiz
- Create all your questions in well organised question bank categories. You can then choose to randomly select questions from the categories, so that each student will get different questions (form formative test only).
- Allow enough time for students to think properly about each question before answering. MCQs take roughly 1 minute to complete, and more complicated ones take 1.5 minutes.
- Provide precise instructions about how long the quiz should take, how many questions it contains, whether negative marking will be applied and the type of questions being asked (Use this Moodle quiz description template).
- Once the quiz is run, go to the results and look at the Navigation > Results > Statistics removing or reworking any that are unreliable - these are the ones that score less than 0 under Discrimination index or Discrimination efficiency columns.
Writing the questions
Determining what to test:
- Test learning outcomes (select, identify, apply theory)
- Focus on important areas of learning and avoid trivia
Writing the question (stem):
- Put the detail in the scenario and question and keep the answers short
- Ensure the stem is not tricky or misleading
Writing the answer:
- Have at least 5 options (this needs to be changed in the quiz settings)
- Ensure all the possible answers are roughly the same length (in badly written MCQs the most complicated answer is often the correct one)
- Add plausible distractors (wrong answers) - you can then give valuable feedback explaining the mistake that was made
- Distractors should relate to one another (i.e. be all metals or all liquids)
- Don't repeat the stem words in the answer(s)
Avoid (in both questions and answers):
- abbreviations
- controversial topics
- absolutes (always, never, all, none)
- may or could (they suggest that anything is possible)
- imprecise terms such as sometimes or rarely - use 15%
What’s good about MCQs?
- Test a wide range of knowledge in a short time
- Potentially reliable (providing a true reflection of student performance) and valid (testing intended learning outcomes)
- Objective and quick to mark
- Automated item analysis possible in Moodle
MCQs - What’s problematic?
- Challenging to write questions which test higher order learning (application, problem-solving)
- Prompts the student – the answer is there, examinees can’t describe or discuss things drawing on their own knowledge
- Difficult and time-consuming to construct flawless MCQs - US estimate one hour per question (Collins)
Example of a single best answer question
Notice that the scenario and question are relatively long and descriptive and the answers are short and plausible.
(Based upon Case and Swanson 2002, p. 38)
