Aronow, Peter M., Alexander Coppock, Forrest W. Crawford, and Donald P. Green. 2015. “Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence.” Journal of Survey Statistics and Methodology. 3(1): 43-66.

Abstract

Survey respondents may give untruthful answers to sensitive questions when asked directly. In recent years, researchers have turned to the list experiment (also known as the item count technique) to overcome this difficulty. While list experiments are arguably less prone to bias than direct questioning, list experiments are also more susceptible to sampling variability. We show that researchers need not abandon direct questioning altogether in order to gain the advantages of list experimentation. We develop a nonparametric estimator of the prevalence of sensitive behaviors that combines list experimentation and direct questioning. We prove that this estimator is asymptotically more efficient than the standard difference-in-means estimator, and we provide a basis for inference using Wald-type confidence intervals. Additionally, leveraging information from the direct questioning, we derive two nonparametric placebo tests for assessing identifying assumptions underlying list experiments. We demonstrate the effectiveness of our combined estimator and placebo tests with an original survey experiment.

Figure

Figure 1 from paper, showing the increase in precision of the combined estimator:

what image shows

Example using list package in R

The combinedListDirect() function in the list package for R will return point estimates and standard error estimates for the direct estimate, the conventional list estimate, and the proposed combined estimator. The summary command will also return the results of Placebo Test I and II.

# uncomment if list package not installed
# install.packages("list"") 

library(list)
# Load data from Aronow, Coppock, Crawford, and Green (2015)
data("combinedListExps")

# complete case analysis
combinedListExps <- na.omit(combinedListExps)
# Conduct estimation without covariate adjustment
out.1 <- combinedListDirect(list1N ~ list1treat, 
                            data = subset(combinedListExps, directsfirst==1), 
                            treat = "list1treat", direct = "direct1")
summary(out.1)
## 
##  Combined List Estimates 
## 
## Call: combinedListDirect(formula = list1N ~ list1treat, data = subset(combinedListExps, 
##     directsfirst == 1), treat = "list1treat", direct = "direct1")
## 
##  Prevalence estimates
##                  Combined     Direct Conventional
## Estimate       0.66561842 0.65600000    0.7477391
## Standard Error 0.04840343 0.02126576    0.0844101
## 
##  Placebo Test I
##        Beta is the conventional list experiment estimate among those who answer 'Yes' to the direct question.
##        Ho: beta = 1
##        Ha: beta != 1
##       
##       Estimate         SE         p   n
## beta 1.053962 0.09452734 0.5680971 328
## 
##  Placebo Test II
##        Delta is the average effect of the receiving the treatment list on the direct question response.
##        Ho: delta = 0
##        Ha: delta != 0
##       
##          Estimate         SE         p   n
## delta 0.06626651 0.04243975 0.1184234 500

Bibtex Citation

@article{aronow201list,
  title={Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence},
  author={Aronow, Peter M. and Coppock, Alexander and Crawford, Forrest W. and Green, Donald P.},
  journal={Journal of Survey Statistics and Methodology},
  pages={smu023},
  year={2015},
  publisher={Oxford University Press}
}