Abstract

Three experiments were conducted to examine the effects of question difficulty order on people’s judgments of test performance and test experiences. Building on the finding that ordering questions from easy to hard often leads to overconfidence (i.e., a retrospective bias), the study aimed to examine the generality and robustness of this effect by having participants from a diverse population take an online test and then make a post-test judgement of their performance. In addition to using the same ascending and descending order of difficulty as prior research, the study also explored how the U-shaped order (e.g., easy-hard-easy) and report option affect such judgments. The results showed that the ordering of question difficulty influenced participants’ judgements of their test performance with each order producing different patterns of bias. It was found that the easy-hard order led to optimism more than the hard-easy order, and the hard-easy-hard led to pessimism more than the easy-hard-easy order. Providing the forced report option reduced judgment bias for the U-shaped order. Findings suggest that the accuracy of participants’ evaluation of their test performance is prone to biases arising from two sources: The cognitive heuristic of anchoring and adjustment while monitoring performance during the test, and primacy and recency effects in which their initial and final testing experience becomes more salient after the test. Additionally, the ordering and report option yielded differential effects on participants’ subjective test experiences.

Committee Chair

Dr. Andrew C. Butler

Committee Members

Dr. Rowhea M. Elmesky, Dr. Christopher S. Rozek

Degree

Master of Arts (AM/MA)

Author's Department

Education

Author's School

Graduate School of Arts and Sciences

Document Type

Thesis

Date of Award

Spring 5-21-2021

Language

English (en)

Author's ORCID

https://orcid.org/0000-0002-7494-9425

Download

Included in

Education Commons

Share

COinS