What the research says

Easy Questions First? What Question Order Does

Put the same questions in a different order and scores stay the same. But students who start with easy questions walk out believing they did better. Here is what that study found.

By the Giga Quizzes team4 min read
Steps painted orange, red and pink climbing against a blue sky
Photo: Viktor Forgacs, Unsplash, Unsplash License, cropped.
In this post
  1. 1Two competing guesses
  2. 2The study
  3. 3What they found
  4. 4Why it matters
  5. 5The limits
  6. 6Ordering a quiz

Anyone who writes a quiz has to decide what goes first. Some teachers like to open with a gentle question to settle nerves. Others put questions in the order the topics were taught, or shuffle them. Yana Weinstein and Henry Roediger at Washington University asked whether the order changes anything, and found an effect that has nothing to do with what students know and everything to do with how they feel about it afterwards.

Two competing guesses

Before running the study, Weinstein described two plausible predictions, each based on older research about how people judge experiences they have just been through.

  • The ending matters most. Research on painful medical procedures had found that people’s memory of the whole experience depends heavily on how it ended. On that logic, a test that ends with hard questions should feel worse, and one that ends with easy questions should feel better.
  • The beginning matters most. Research on how we form impressions of other people had found that the first things we learn about someone weigh more than the later ones. On that logic, a test that starts with hard questions should feel worse.

The two predictions point in opposite directions for the same test, which makes the question a neat one to answer.

The study

Participants answered a set of 50 general-knowledge trivia questions whose difficulty had been measured in advance. The questions were taken from an established set of norms from 1980, and because general knowledge changes over thirty years, the researchers piloted them again to check how hard each one now was.

The same 50 questions were then given in different orders to different groups. One group saw them from easiest to hardest, another from hardest to easiest. Early experiments in the series also included a random order. After the test, every participant estimated how well they had done.

What they found

Actual scores were the same whatever the order. That makes sense: the questions were identical, and knowing the answer to a trivia question does not depend on what came before it.

But the estimates were not the same. Participants who started with the easy questions were consistently more optimistic about how they had done than those who started with the hard ones, even though their scores matched. The beginning won. The first few questions set an impression of the test, and that impression coloured the judgement of the whole thing, as the title of the paper puts it: easy items at the beginning make students believe they did better.

The authors call this a retrospective bias. It is not about how students perform while answering but about how they remember and judge the test once it is over. Later work by the same researchers looked at how the bias develops during a test, and other studies tested whether it can be removed.

Why it matters

If order does not change scores, why care? Because how students judge a test affects what they do next. A student who walks out believing it went badly may study differently, feel less able, or approach the next test more anxiously, all on the basis of an impression created by the first few questions rather than by what they actually knew.

The bias also affects teachers. A class that has just finished a quiz starting with three hard questions may be gloomier than its results justify, and the mood in the room is not a reliable guide to the results.

The limits

The studies used general-knowledge trivia rather than material from a course the participants were taking, and the judgement in question was a single estimate after the test. Order can matter in other ways too. On a real exam, hard questions at the start can eat time that students then lack for easier ones later, which is a practical problem this research was not designed to study. And these were the two extreme orders; most real tests mix difficulties.

Ordering a quiz

For a classroom quiz, where the point is learning and confidence rather than ranking students, the finding gives a clear default.

A sensible running order

  • Open with one or two questions almost everyone can answer.
  • Build up to the hardest questions in the middle.
  • Close with something students can feel good about, if you can.
  • When students estimate their own score, remind them how the test began.
  • For real assessment, keep the order the same for everyone so judgements are comparable.

An easy opener has a second benefit in a live game: it gets every player answering, and scoring, from the first question, so nobody starts the game feeling left behind. Feeling able to succeed is one of the things research on motivation in classroom quizzes keeps returning to. Starting easy does not make the quiz easier overall; it only changes the first impression, and the evidence says the first impression is the one that lasts.

Sources

  1. Weinstein, Y. & Roediger, H. L. (2010). Retrospective bias in test performance: Providing easy items at the beginning of a test makes students believe they did better on it. Memory & Cognition, 38(3), 366–376.
  2. Weinstein, Y. (2018). How does question difficulty order affect evaluations of test performance? (parts 1 and 2). The Learning Scientists blog.

More from the blog

All posts →