Matthew Graham and Alexander Coppock (2020), “Asking about Attitude Change.” Working paper.

Abstract

Surveys often ask respondents how information or events changed their attitudes. Does [information X] make you more or less supportive of [policy Y]? Does [scandal X] make you more or less likely to vote for [politician Y]? We show that this type of question (the change format) exhibits poor measurement properties, in large part because subjects engage in response substitution. When asked how their attitudes changed, people often appear to report the level of their attitudes rather than the change in them. As an alternative, we propose the counterfactual format, which asks subjects what their attitude would have been in the counterfactual world in which they did not know the treatment information. Using a series of experiments embedded in four studies, we show that the counterfactual format greatly reduces bias relative to the change format.

Figure

Figure 1 from paper, showing the counterfactual format estimates of the effect of the Ukraine revelations on support for impeachment.

Graham and Coppock Figure 1