What the research says

Three Answer Options Are Enough

Four or five options per multiple-choice question is the habit, but 80 years of research say three usually do the job. A study of 514 real test questions shows why the fourth rarely works.

By the Giga Quizzes team4 min read
Three pink ice lollies lying side by side among berries
Photo: Aron Visuals, Unsplash, Unsplash License, cropped.
In this post
  1. 1Eighty years of evidence
  2. 2Why the extra options do not work
  3. 3What about guessing?
  4. 4The limits
  5. 5Writing three good options

Most multiple-choice questions have four options, and many exams insist on five. Anyone who has written them knows the problem: the right answer comes easily, the first wrong answer comes quite easily, and by the third wrong answer you are inventing something nobody would ever pick. Research on multiple-choice tests has been looking at this for a very long time, and its answer is consistent. For most questions, three options are as good as four or five, and sometimes better.

Eighty years of evidence

In 2005 Michael Rodriguez, a specialist in educational measurement, brought together the studies that had compared questions with different numbers of options, going back about 80 years. His conclusion is in the title of the paper: three options are optimal.

He looked at the things test designers care about. How hard the question is. How well it separates students who know the material from those who do not, which is called discrimination. And how reliable the whole test is, meaning how consistently it ranks students. Going from four or five options down to three did not harm the quality of the test scores. And because three-option questions are quicker to read and answer, a test of the same length can include more of them, which covers more of the course.

Rodriguez also points out that researchers have recommended three options for over eight decades, and that the habit of four or five has survived anyway.

Why the extra options do not work

A study by Marie Tarrant, James Ware and Ahmed Mohammed shows what happens in practice. They analysed seven real exams from a nursing degree in Hong Kong, written by the lecturers themselves, with 514 four-option questions and 1,542 wrong options in all.

They counted a wrong option as working only if at least 5% of students picked it, and if it attracted weaker students more than stronger ones. An option nobody chooses adds nothing. An option that strong students choose is usually a sign that the question is ambiguous.

Working wrong options per questionShare of questions
None12.3%
One34.8%
Two39.1%
All three13.8%
Tarrant, Ware & Mohammed (2009), 514 four-option questions

Only about one question in seven had three wrong options that all did their job. On average, a question had 1.54 working wrong options. Only 52% of all the wrong options were functioning, and 10% were never chosen by a single student. In other words, most of these “four-option” questions were already two- or three-option questions in disguise, with a dead option or two taking up space.

The researchers then simulated three-option versions of the exams, removing the least chosen wrong option from each question and sharing out the students who had picked it among the rest. The average score went up by about one point, from 65.9% to 66.9%, and reliability barely moved, from 0.76 to 0.75.

What about guessing?

The obvious worry is that three options make guessing easier: a blind guess is right one time in three instead of one in four. In theory that is true. In practice, students rarely guess blind. They rule out options they know are wrong and guess between the rest. A fourth option that everyone rules out instantly does nothing against guessing; it just costs reading time. A good third option does more than a weak third and fourth together.

Tarrant and her colleagues add a sensible rule of thumb. There is no reason every question in a test must have the same number of options. Write as many good wrong options as the content allows, and stop there.

The limits

Most of the evidence comes from formal tests at school, university and professional level, analysed for how well they measure. Classroom quizzes have other jobs as well, such as practice and discussion, and these studies did not measure learning. The Hong Kong study looked at one nursing programme, though its pattern matches earlier reviews of professionally written questions. And some questions genuinely have four strong options, in which case there is no reason to cut one.

Writing three good options

The real lesson is that the quality of the wrong answers matters far more than their number. A tempting wrong answer is what makes a question worth asking.

Finding wrong answers that work

  • Use the mistakes students actually make: the common misconception, the step people skip, the near miss.
  • Make every option the same length and style, so the right one does not stand out.
  • Drop any option you added only to reach four.
  • After a quiz, look at which wrong answers nobody chose, and replace or remove them next time.
  • Use four options when you genuinely have three convincing wrong answers, and not otherwise.

Fewer, better options also make a live quiz faster to read on a small phone screen, which leaves more time for thinking about the question itself. For more on what makes an option misleading in the wrong way, see common quiz writing mistakes and true or false questions.

Sources

  1. Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3–13.
  2. Tarrant, M., Ware, J. & Mohammed, A. M. (2009). An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis. BMC Medical Education, 9, 40.

More from the blog

All posts →