Date of Award
Master of Arts (AM/MA)
Three experiments were conducted to examine the effects of question difficulty order on people’s judgments of test performance and test experiences. Building on the finding that ordering questions from easy to hard often leads to overconfidence (i.e., a retrospective bias), the study aimed to examine the generality and robustness of this effect by having participants from a diverse population take an online test and then make a post-test judgement of their performance. In addition to using the same ascending and descending order of difficulty as prior research, the study also explored how the U-shaped order (e.g., easy-hard-easy) and report option affect such judgments. The results showed that the ordering of question difficulty influenced participants’ judgements of their test performance with each order producing different patterns of bias. It was found that the easy-hard order led to optimism more than the hard-easy order, and the hard-easy-hard led to pessimism more than the easy-hard-easy order. Providing the forced report option reduced judgment bias for the U-shaped order. Findings suggest that the accuracy of participants’ evaluation of their test performance is prone to biases arising from two sources: The cognitive heuristic of anchoring and adjustment while monitoring performance during the test, and primacy and recency effects in which their initial and final testing experience becomes more salient after the test. Additionally, the ordering and report option yielded differential effects on participants’ subjective test experiences.
Chair and Committee
Dr. Andrew C. Butler
Dr. Rowhea M. Elmesky, Dr. Christopher S. Rozek
Fang, Wei-Chieh, "The Effects of Question Difficulty Order on Metacognitive Judgments During an Online Test" (2021). Arts & Sciences Electronic Theses and Dissertations. 2280.
Available for download on Tuesday, April 25, 2023