When to Worry About Sensitivity Bias: Evidence from 30 Years of List Experiments
Blair, Graeme, Alexander Coppock, Margaret Moor. 2020. When to Worry About Sensitivity Bias: Evidence from 30 Years of List Experiments. American Political Science Review.
Abstract
Eliciting honest answers to sensitive questions is frustrated if subjects withhold the truth for fear that others will judge or punish them. The resulting bias is commonly referred to as social desirability bias, a subset of what we label sensitivity bias. We make three contributions. First, we propose a social reference theory of sensitivity bias to structure expectations about survey responses on sensitive topics. Second, we explore the bias-variance tradeoff inherent in the choice between direct and indirect measurement technologies. Third, to estimate the extent of sensitivity bias, we meta-analyze the set of published and unpublished list experiments (a.k.a., the item count technique) conducted to date and compare the results with direct questions. We find that sensitivity biases are typically smaller than 10 percentage points and in some domains are approximately zero.
Links
Figure
