Question Order Effects Decrease with Age

Question order effects emerge when preceding questions bring information to mind that respondents may otherwise not consider in answering a subsequent question (see Sudman et al., 1996, for a discussion of the underlying processes). To reduce these effects, researchers often introduce buffer items to separate related questions, thus attenuating the accessibility of previously used information. The same logic suggests that age-related declines in memory function attenuate question order effects because they render it less likely that previously used information is still accessible. The available data support this prediction (Knäuper, Schwarz, Park, and Fritsch, 2003, unpublished manuscript; Schwarz and Knäuper, 2000).

One of the most robust question order effects in the survey literature was observed by Schuman and Presser (1981), who asked respondents if a pregnant woman should be able to obtain a legal abortion “if she is married and does not want any more children” (Question A) or “if there is a strong chance of a serious defect in the baby” (Question B). Not surprisingly, more respondents support legal abortion in response to Question B than in response to Question A. More important, support for Question A decreases when Question B is presented first: Compared to the risk of a serious birth defect, merely “not wanting any more children” appears less legitimate, reducing support for legal abortion. Secondary analyses show that this question order effect results in a difference of 19.5 percentage points for younger respondents, but decreases with age and is no longer observed for respondents aged 65 and older (Figure 1). Thus researchers arrive at different conclusions about cohort differences depending on the order in which the questions are asked. When the “no more children” question (A) is asked first, support for abortion decreases with age, suggesting that older respondents hold more conservative attitudes toward abortion. Yet no age difference can be observed when the same question is preceded by the “birth defect” question (B).

FIGURE 1

Question order effects as a function of age. NOTE: Columns show percentages of respondents who support abortion for “a married woman who doesn't want any more children” when this question precedes (A-B) or follows (B-A) the birth defect (more...)

Laboratory experiments (Knäuper et al., 2003) indicate that the attenuation of question order effects among older respondents is due to age-related declines in working memory capacity. Using Salthouse and Babcock's (1991) reading span test, Knäuper and colleagues (2003) found that younger respondents (age 20 to 40 years), as well as older respondents (age 60 to 100 years) with good working memory capacity, showed the familiar question order effect, whereas older respondents with poor working memory capacity did not (Figure 2), suggesting that the attenuation of question order effects among older respondents is due to age-related declines in working memory capacity. A related experiment with different questions replicated this pattern.

FIGURE 2

Question order effects as a function of age and working memory. NOTE: Columns show the percentage of respondents who support abortion for “a married woman who doesn't want any more children” (question A) versus the percentage of respondents (more...)

However, the observed attenuation of question order effects may be limited to the relatively uninvolving questions typical for public opinion surveys. In contrast, older adults' inhibition problems (Hasher and Zacks, 1988) may exaggerate question order effects when the question is of high personal relevance or emotionally involving. This possibility remains to be tested.

Response Order Effects Increase with Age

A second major source of context effects in attitude measurement is the order in which response alternatives are presented. Response order effects are most reliably obtained when a question presents several plausible response options (see Schwarz, Hippler, and Noelle-Neumann, 1992; Sudman et al., 1996, Chapter 6, for detailed discussions). To understand the underlying processes, suppose you were asked to provide a few reasons why “divorce should be easier to get.” You could easily do so, yet you could just as easily provide some reasons why “divorce should be more difficult to get.” When such alternatives are juxtaposed (as in “Should divorce be easier to get or more difficult to get?”), the outcome depends on which alternative is considered first. While the researcher hopes that respondents (a) hold the question and all response alternatives in mind, (b) evaluate the implications of each alternative, and (c) finally select the one that they find most agreeable, respondents may not do so. Instead, respondents who first think about “easier” may come up with a good supporting reason and may endorse this answer without engaging in the additional steps.

Importantly, the likelihood that respondents elaborate on a given alternative depends on the order and mode in which the response options are presented (Krosnick and Alwin, 1987). When presented in writing, respondents elaborate on the implications of the response options in the order presented. In this mode, an alternative that elicits supporting thoughts is more likely to be endorsed when presented early rather than late on the list, giving rise to primacy effects. In contrast, when the alternatives are read to respondents, their opportunity to think about the early ones is limited by the need to listen to the later ones. In this case, they are more likely to work backward, thinking first about the last alternative read to them. When this alternative elicits supporting thoughts, it is likely to be endorsed, giving rise to recency effects. Thus, a given alternative is more likely to be endorsed when presented early rather than late in a visual format (primacy effect), but when presented late rather than early in an auditory format (recency effect).

On theoretical grounds, we may expect that older respondents find it more difficult to keep several response alternatives in mind, elaborating on their respective implications to select the most appropriate answer. This should be particularly true in telephone interviews, where the alternatives are read to respondents without external memory support. The available data strongly support this prediction (see Knäuper, 1999, for a comprehensive review and meta-analysis). For example, Schuman and Presser (1981) asked respondents in a telephone interview, “Should divorce in this country be easier to obtain, more difficult to obtain, or stay as it is now?” Depending on conditions, the response alternative “more difficult” was read to respondents as the second or as the last alternative. Overall, respondents were somewhat more likely to select the response alternative “more difficult” when presented last, a recency effect. However, secondary analyses reported by Knäuper (1999) indicate a dramatic age difference: As shown in Figure 3, the size of the recency effect increased with respondents' age, ranging from a nonsignificant 5 percent for age 54 and younger to 36.4 percent for age 70 and older. Thus different substantive conclusions will be drawn about the relationship of age and attitudes toward divorce, depending on the order in which the response alternatives are presented: whereas attitudes toward divorce seem to become much more conservative with age under one order condition, no reliable age differences are obtained under the other order condition.

FIGURE 3

Response order effects as a function of age. NOTE: Columns show percentage of respondents who endorse the statement that “divorce should be more difficult to get” when this alternative is presented as the second or the last of three options (more...)

The available data suggest that age differences in response order effects are limited to the more taxing auditory format and are not observed when all response alternatives are presented in writing and remain visible. As an example, consider a study in which younger (age 20 to 40) and older (age 65+) adults were asked, “Which of the following four cities do you find most attractive?” (Knäuper et al., 2003). Washington, DC, was presented as either the first or last choice (Table 1).

TABLE 1

Age, Mode, and Response Order Effects.

When the response alternatives were read to respondents, younger as well as older adults were more likely to choose Washington, DC, when it was presented last rather than first. Replicating the pattern of the divorce findings, this recency effect was more pronounced for older than younger respondents, with differences of 24 versus 8 percentage points. When the response options were presented in writing, however, older as well as younger respondents were more likely to choose Washington, DC, when it was presented first rather than last. Importantly, this primacy effect under a visual mode was of comparable size for both age groups.

Summary

In sum, the reviewed findings suggest that question order effects are likely to decrease with age, whereas response order effects are likely to increase with age. Both of these effects can be traced to age-related declines in cognitive resources. Hence, self-reports of attitudes are not only context dependent, but the size of the emerging context effects is itself age sensitive, rendering comparisons across age groups fraught with uncertainty. In addition, several studies indicate that older respondents are more likely to provide a “don't know” or “no opinion” answer (for a review see Schwarz and Knäuper, 2000). The processes underlying the latter observation are still unknown.

Subjective Theories

A particularly important estimation strategy is based on subjective theories of stability and change (see Ross, 1989, for a review). In answering retrospective questions, respondents often use their current behavior or opinion as a benchmark and invoke an implicit theory of self to assess whether their past behavior or opinion was similar to, or different from, their present behavior or opinion. Assuming, for example, that one's political beliefs become more conservative over the life span, older adults may infer that they held more liberal political attitudes as teenagers than they do now (Markus, 1986). The resulting reports of previous opinions and behaviors are correct to the extent that the implicit theory is accurate. In many domains, however, individuals assume a rather high degree of stability, resulting in underestimates of the degree of change that has occurred over time (Collins, Graham, Hansen, and Johnson, 1985; Withey, 1954), while in other domains (Ross, 1989) respondents who believe in change will detect a change even though none has occurred.

At present, little is known about whether and how subjective theories of stability and change are themselves subject to change across the life span: Which aspects of the self do people perceive as stable versus variable? And at which life stage do they expect changes to set in? Moreover, it is likely that salient life events, like retirement or the loss of a spouse, will give rise to subjective theories of profound change. If so, retrospective reports pertaining to earlier time periods may exaggerate the extent to which life was different prior to the event, conflating real change with biased reconstructions. These issues provide a promising avenue for future research.

Response Alternatives

An alternative estimation strategy draws on the frequency scale presented by the researcher (for a review see Schwarz, 1999a). In a nutshell, respondents assume that the researcher constructs a meaningful scale. Hence, values in the middle range of the scale presumably reflect the “average” or “typical” behavior, while values at the extremes of the scale correspond to the extremes of the distribution. Based on this assumption, respondents use the frequency scale as a frame of reference in estimating the frequency of their own behaviors. This results in higher frequency reports along scales with high rather than low values. For example, 37.5 percent of a sample of German respondents reported watching TV for more than 2 ½ hours a day when given a high frequency ranging from “up to 2 ½ hours” to “more than 4 hours” a day. In contrast, only 16.2 percent reported doing so when given a low frequency scale ranging from “up to ½ an hour” to “more than 2 ½ hours” (Schwarz, Hippler, Deutsch, and Strack, 1985).

Such scale-based estimation effects are more pronounced the more poorly the behavior is represented in memory (Menon, Rhagubir, and Schwarz, 1995). This suggests that the impact of response alternatives may typically be more pronounced for older than for younger respondents. The available data support this prediction, with some important qualifications. As shown in Table 2, Knäuper, Schwarz, and Park (2004) observed that older respondents were more affected by the frequency range of the response scale when asked to report on the frequency of mundane events, such as buying a birthday present. On the other hand, older respondents were less affected than younger respondents when the question pertained to the frequency of physical symptoms, which older respondents are more likely to monitor (e.g., Borchelt, Gilberg, Horgas, and Geiselmann, 1999). In combination, these findings suggest that respondents of all ages draw on the response alternatives when they need to form an estimate of how often they do something. Yet the need to estimate depends on how much attention they pay to the respective behavior, and this attention is itself age dependent.

TABLE 2

Impact of Response Alternatives on Behavioral Reports as a Function of Content and Respondents' Age.

Note that the scale format used leads to different conclusions about age-related differences in actual behavior from these reports. For example, age differences in red meat consumption (a health-relevant dietary behavior) or the purchase of birthday presents (an indicator of social integration) appear to be minor when a low frequency scale is used, but rather pronounced when a high frequency scale is used. To avoid systematic influences of response alternatives, and the age-related differences in their impact, it is advisable to ask frequency questions in open response formats that specify the relevant units of measurement. While the reports obtained under an open format are far from error free, they are at least not systematically biased by the instrument. See Schwarz (1999a) for a discussion.