Tips for developing good Multiple Choice Questions (MCQs)

What are Multiple Choice Questions?

Multiple choice items consist of a stem and a set of options. The stem is the beginning part of the item that presents the item as a problem to be solved, a question asked of the respondent, or an incomplete statement to be completed, as well as any other relevant information. The options are the possible answers that the examiner can choose from, with the correct answer called the key and the incorrect answers called distractors (http://en.wikipedia.org/wiki/Multiple_choice).

Tips for setting up and fine-tuning a Moodle quiz

Create all your questions in well organised question bank categories. You can then choose to randomly select questions from the categories, so that each student will get different questions (form formative test only).
Allow enough time for students to think properly about each question before answering. MCQs take roughly 1 minute to complete, and more complicated ones take 1.5 minutes.
Provide precise instructions about how long the quiz should take, how many questions it contains, whether negative marking will be applied and the type of questions being asked (Use this Moodle quiz description template).

Once the quiz is run, go to the results and look at the Navigation > Course > Quiz name > Results > Statistics. Remove or rework any questions that are unreliable - these are the ones that score less than 0 under Discrimination index or Discrimination efficiency columns.

Writing the questions

Determining what to test:

Test learning outcomes (select, identify, apply theory)
Focus on important areas of learning and avoid trivia

Writing the question (stem):

Put the detail in the scenario and/or question and keep the answers short
Ensure the stem is not tricky or misleading

Writing the answer:

Have at least 5 options (this needs to be changed in the quiz settings)
Ensure all the possible answers are roughly the same length (in badly written MCQs the most complicated answer is often the correct one)
Add plausible distractors (wrong answers) - you can then give valuable feedback explaining the mistake that was made
Distractors should relate to one another (i.e. be all metals or all liquids)
Don't repeat the stem words in the answer(s)

Avoid (in both questions and answers):

abbreviations
controversial topics
absolutes (always, never, all, none)
may or could (they suggest that anything is possible)
imprecise terms such as sometimes or rarely - use 15%

What’s good about MCQs?

Test a wide range of knowledge in a short time
Potentially reliable (providing a true reflection of student performance) and valid (testing intended learning outcomes)
Objective and quick to mark
Automated item analysis possible in Moodle

MCQs - What’s problematic?

Challenging to write questions which test higher order learning (application, problem-solving)
Prompts the student – the answer is there, examinees can’t describe or discuss things drawing on their own knowledge
Difficult and time-consuming to construct flawless MCQs - US estimate one hour per question (Collins)

Example of a single best answer question

Notice that the scenario and question are relatively long and descriptive and the answers are short and plausible.

(Based upon Case and Swanson 2002, p. 38)

Aknowledgments

Thank you to Ros Duhs from UCL Centre for the Advancement of Learning and Teaching (CALT) who provided much of the advice on this page.

References

Case S.M. and Swanson D.B. (2002) Constructing written test questions for the basic and clinical sciences. Philadelphia, PA: National Board of Medical Examiners. http://www.nbme.org/PDF/ItemWriting_2003/2003IWGwhole.pdf

Collins, J. (n.d.) Writing Multiple Choice Questions for Continuing Medical Education Activities and Self-Assessment Modules. http://www.arrs.org/uploadedFiles/ARRS/Publications/writingMultipleChoiceHandout.pdf