What the research says

Fact Questions or Thinking Questions?

Do students need the facts before they can tackle harder questions? A 2019 study quizzed students on facts, on higher-order questions or on a mix, and the mix came out on top.

By the Giga Quizzes team4 min read
A glowing light bulb against a pink and orange background
Photo: Jakub Żerdzicki, Unsplash, Unsplash License, cropped.
In this post
  1. 1Two kinds of question
  2. 2In the laboratory
  3. 3In the classroom
  4. 4The limits
  5. 5Writing a mixed quiz

A common reading of Bloom’s taxonomy goes like this: first students must know the facts, and only then can they apply, analyse and evaluate. So quizzes should start with recall questions and move up the pyramid later. It sounds sensible, and almost nobody had tested it. Pooja Agarwal did, in three experiments published in 2019, two in a university laboratory and one in a real sixth-grade classroom. The results do not support the step-by-step version of the pyramid.

Two kinds of question

In the classroom experiment, the material was two chapters of a social studies textbook, on the Russian Revolution and on the Second World War. Here are two questions from the paper, both multiple choice.

A fact question

Why was Nicholas II forced to give up his role as tsar? A. Because the Duma elected a new tsar. B. Because Stalin took over the government. C. Because his wife and children moved to Moscow. D. Because of angry protestors, soldiers, and railroad workers. (D is correct, and the answer was stated directly in the chapter.)

A higher-order question

Which person would agree with the following statement? “Revolutions are hard to prevent.” A. Alexander II. B. Lenin. C. Nicholas II. D. Stalin. (C is correct, but the chapter never says so. You have to work it out from what happened to him.)

The fact questions were not trivia. Agarwal deliberately avoided names, dates and definitions and asked about the key ideas of each chapter. The higher-order questions asked students to apply, analyse or evaluate those ideas.

In the laboratory

In the first experiment, 48 university students read short passages once, read them twice, or read them and then took a quiz of fact questions or of higher-order questions. Two days later they took a test with both kinds of question.

Fact quizzes helped on the fact test: 78% against 54% for students who had only read. But they barely helped on the higher-order test: 46%, against 44% and 49% for reading once and twice. Higher-order quizzes helped on the higher-order test, 72%. Practice transferred to the kind of question that had been practised, and building the fact base first did not lift students up the pyramid.

The second experiment added a mixed quiz, half fact and half higher-order. It produced the highest overall delayed score, 75%, against 69% for fact quizzes or higher-order quizzes alone.

That experiment also ruled out a simple explanation. Perhaps the mixed group did well just because it had more practice. But two higher-order quizzes scored 69%, only a little above one higher-order quiz at 65%, and still well below the mix. What mattered was the combination of question types, not the amount of quizzing.

In the classroom

The third experiment ran with 142 sixth graders in six classes at a suburban middle school, using the teacher’s own chapters. Each topic got three clicker quizzes, before the lesson, after it and as a review, made either of higher-order questions or of a mix of fact and higher-order questions. Some content was not quizzed at all. Two days after the review quiz, students took a test of both kinds.

The quizzes worked as teaching on their own terms. Students averaged 38% on the quiz before each lesson, 71% just after it and 84% on the review, helped along by the lesson, the textbook and the answer shown after every clicker question.

Two days laterFact testHigher-order testAverage
Not quizzed64%56%60%
Higher-order quizzes64%75%70%
Mixed quizzes91%82%86%
Agarwal (2019), Experiment 3

Mixed quizzes were best on both tests. On the higher-order test they beat higher-order quizzes alone, even though the higher-order group had practised exactly that kind of question. Fact questions alone were not tested in this classroom, so the result does not mean fact quizzes are useless. It means that adding thinking questions to them did a great deal more.

The limits

The tests came two days after practice, so long-term effects are unknown. The classroom experiment took place in one school with one teacher, and with sixth graders on history content; younger children or very unfamiliar material may need more fact work first. The questions were all multiple choice. And the higher-order questions were carefully written, which is harder than it looks. A badly written thinking question is often just a confusing fact question.

Writing a mixed quiz

The practical message is not to abandon fact questions. It is to stop treating higher-order questions as a reward for later, once the basics are secure. Students can practise both from the start, and the two seem to help each other.

Turning a fact into a thinking question

  • Apply: give a new example and ask which idea it shows.
  • Analyse: ask which person, event or object fits a statement the text never makes directly.
  • Evaluate: ask which of several arguments is strongest, and why.
  • Keep about half the quiz as straightforward recall of the key ideas.
  • After each thinking question, say which fact it rested on.

Multiple choice can carry higher-order questions perfectly well, as both examples above show. For more on writing good options, see common quiz writing mistakes and turning a lesson into quiz questions.

Sources

  1. Agarwal, P. K. (2019). Retrieval practice & Bloom’s taxonomy: Do students need fact knowledge before higher order learning?. Journal of Educational Psychology, 111(2), 189–209.

More from the blog

All posts →