Do Evaluations Rise With Experience?

Psychol Sci. 2018 May;29(5):779-790. doi: 10.1177/0956797617744517. Epub 2018 Mar 1.

Abstract

Sequential evaluation is the hallmark of fair review: The same raters assess the merits of applicants, athletes, art, and more using standard criteria. We investigated one important potential contaminant in such ubiquitous decisions: Evaluations become more positive when conducted later in a sequence. In four studies, (a) judges' ratings of professional dance competitors rose across 20 seasons of a popular television series, (b) university professors gave higher grades when the same course was offered multiple times, and (c) in an experimental test of our hypotheses, evaluations of randomly ordered short stories became more positive over a 2-week sequence. As judges completed repeated evaluations, they experienced more fluent decision making, producing more positive judgments (Study 4 mediation). This seemingly simple bias has widespread and impactful consequences for evaluations of all kinds. We also report four supplementary studies to bolster our findings and address alternative explanations.

Keywords: experience; judgment and decision making; open data, open materials; processing fluency; sequential evaluation.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adult
  • Decision Making / physiology*
  • Humans
  • Judgment / physiology*
  • Practice, Psychological*
  • Time Factors